The text below has been anonymised. The exact names of people and companies are irrelevant. The blog is written to share lessons learned. The phone rings. It is the end of September 2022. I answer and I hear the voice of an old acquaintance. Let’s call him John. After the usual “hey, how are you?” it becomes clear why Jan is calling. He had an IT platform developed for his company by “a friend with four Indians in India”. There has now been a huge argument between the parties and it is possible that it will turn into a lawsuit. Meanwhile, that “friend” threatens to pull the plug while the platform is the lifeline of Jan’s company. “You have an IT company, don’t you? Can you take over then?” It should be said that the platform was already live, but had many hiccups and a huge list of functional changes, small and very large, was ready.
That’s what I like! Helping someone in need, especially an old acquaintance. And doing something that many companies are hesitant about: taking over a platform built by someone else, without even knowing anything about the quality of that code, the documentation present, the built-in logging and the monitoring implemented. Yes, THAT is a cool challenge!
We got to work. Initially secretly, because the argument with Jan’s “friend” was still in full swing and he was not allowed to know anything. In his anger, he might decide to really turn the switch on, to turn things off. We had to act “under the radar” for the time being.
We set up the team according to the devops principle so that, once we had mastered the code, we could carry out the operational management of the platform as well as add improvements and new functionality to the platform.
Of course we needed the login details for AWS, where everything ran. Jan did not have it and had to request it from his “friend”. Also, of course, we wanted the source code. John didn’t have that either. Requested again, and with some fits and starts it became available. Little by little we were able to eat ourselves in.
We soon realized that the code did not quite meet our quality level. And then I express myself very carefully. So be it. We had to deal with it anyway. We saw in various places that it was not properly arranged in terms of security of API calls, etc. We immediately went to fix that anyway. We also discovered some backdoors here and there. Pieces of code that, in our opinion, were not built in because they were functionally or technically necessary, but that may have been built in to be accessible in case of a fight. Hopsakee, those pieces of code had to be removed. Quite fascinating to then start cutting into someone else’s spaghetti, and to experience experimentally where the strings all go …
It must be said: Jan understood that we needed time. Okay, initially he said “that will take you two weeks, right?” Logical, nothing human was strange to Jan But after I had rolled off the table from the shaking belly, Jan realized that it would be more like two months. We got that time. Which incidentally did not prevent Jan from indicating every week that so many functional changes still had to be made, that there was a rush, and … Anyway, an enthusiastic client, what more could you want?
Very quickly we decided to rebuild everything from scratch. At least, not that we were going to rewrite complete code because that was impossible. But we did go through the source code line by line, and sometimes made some adjustments. Putting some structure in the spaghetti couldn’t hurt. And we wanted to make sure that we understood the platform from A to Z before we took it into management and started further development. Through reverse engineering we found out that the “friend” had not transferred all the code to Jan. Growl! The missing code was still supplied.
We completely rebuilt the AWS environment on which the platform ran. This way we could be sure that everything was secure, that no back doors were built into the infrastructure and that there would not be a hidden process running somewhere that…
In preparation for the go-live of the slightly cleaned up code on the AWS infrastructure we manage, we have drawn up a detailed cut-over plan to migrate the entire environment in a controlled manner with minimal outage. Between Christmas and New Year’s Eve 2022 the time had come. The platform ran, with all the trimmings, in a new AWS environment, with the existing data that had been migrated from the old database. Just a little moment of joy and then …we started to process the pile of change requests. A list of more than eighty items. But how about the priorities in that list? They were not clear, so we had to do something about that.
How did that go on? To be continued in next week’s blog.