Some of you may have wondered how the wonderful software that powers Yield Manager is created. A lot of people imagine that the people who do the tech work at RightMedia are geeks who wear pocket protectors and sit in front of terminals in dark rooms. Well, that’s mostly true (I have a very stylish pocket protector by the way). But we’re people too. We struggle with problems, make mistakes and then try to blame them on the new guy just like you do. I’m going to try to walk you through the process of solving a problem and implementing a solution.
The Problem: Finding Out Who Dunnit
A lot of clients had been asking for a way to view who had made what changes to a given line item, and for some information about what those changes were. Ed Kozek (director of UI development) tasked me to implement a solution. We saw two potential ways to solve the problem:
1) Store a message in the system anytime that someone made a change involving line item and display it to users
2) Parse the records in the database to try and find changes involving a line item and information about the change
Yield Manager is a very complex application. There are hundreds of different files and thousands of places within those files where someone could make a change. So instead of spending time going through all of these files and trying to add messages, we decided to parse the database logs so that we could better spend that time drinking coffee and playing video games online. (Note to Ed: that was a joke.). Seriously, we decided that our resources would be better spent working on the many features that have a more direct impact on providing business value to our clients. So I was left with what seemed like a relatively straight-forward task. We already log every query that gets run on the database. All I had to do was turn that record into something “human readable”. Simple right?
The Plan: “It All Seemed So Simple Then”
In our database we store the actual SQL queries used to make changes, like these…
DELETE FROM offer_type_table WHERE id=’2918′
INSERT INTO offer_type_table (id, offer_type_id) VALUES (2918, 1)
INSERT INTO offer_type_table (id, offer_type_id) VALUES (2918, 29)
INSERT INTO offer_type_table (id, offer_type_id) VALUES (2918, 33)
…and I have to turn them into (somewhat) friendly text like…
“Joel Hoard modified offer type targeting - Set to exclude: Adult, Mixed/unknown, Casino/Gambling”
Not fun, but I’m a developer right? I mean, this is what we do. I’m going to try to spare you the boring technical details and stick to the boring process details. Our initial plan seemed to be holding up…take the query statements and turn them into text. I spent about four hours getting to that point and went to check how things were coming along. I managed to turn the four statements above into:
Joel Hoard removed offer_type_table targeting.
Joel Hoard modified offer_type_table targeting - Adult
Joel Hoard modified offer_type_table targeting - Mixed/unknown
Joel Hoard modified offer_type_table targeting - Casino/Gambling
So far, not so good.
This was still far off from my goal of: “Joel Hoard modified offer type targeting - Set to exclude: Adult, Mixed/unknown, Casino/Gambling”
After I had played around with it, I realized that the statements in the database didn’t have a one-to-one relationship with the actual changes made by users. I also had to deal with turning table names into names of things that people could recognize. And I had to figure out if people were including or excluding. My simple task had become something of a bigger problem, and all of our SmartPeople(tm) were using their brain power to decipher the mysteries of other, more mission critical tasks. So I had to go back to the drawing board a bit.
The Issues: “Houston, we have a problem.”
I’m glancing over the intricacies of turning a SQL INSERT statement into text of any kind…it was not easy. It’s not the kind of thing that’s covered in college textbunqiue
unqiueunqiueooks or certification courses. I also skipped over the details of gathering and organizing all of the data and all the different database tables I used to look up information.
I did this for two reasons. First, to spare you the “boring details” (though I’m sure this article is boring enough already). And secondly and most important, because the obscure technical references and jargon that we use are our primary means of keeping our jobs. If I explained the details of solving these technical problems, then you would understand that there is no “Magic” in computer programming…just a careful application of logical rules and experience. We programmers like to utilize what we call “Shock & Awe” to make something that is actually relatively straight forward seem confusing and magical. Then you don’t question us when we say that something simple will take a week. (Heh. Just kidding, Ed.)
So I came up with a system to try and match up the groups of database commands by the table they dealt with and the time that the change occured. This is not an exact science. In fact it’s not science at all. It’s more like a sleight-of-hand show. The whole thing is what we call a “hack”: a solution put in place because we don’t have the resources available to do it the “proper” way.
I can turn “offer_type_table” and “frequency_table” into “offer type” and “frequency” by removing “_table”. I can do a decent job of grouping changes together (with 98% or so accuracy) and I can look up the include/exclude information based on the names of the tables involved. But the problems arise when the table involved isn’t “frequency_table” but “frequency_target_table” or “table_frequencies”. Or when the include/exclude information is stored in table A for one type of change and table B for another type of change. Since there are no hard rules to work with, all I could do was work with general rules and add solutions for unique situations as I see them. And for every one unique problem I fix, someone submits a bug report with two new ones. I was trying to be careful to not show INACCURATE information, which can do more harm than no information at all, while still trying to provide a workable solution.
The (Partial) Solution: “Run away! Run away!”
After about a week of this, Ed put an end to the madness. We had a solution that does a decent job of showing history. At least there are dates and names, though the text may not provide any value. This was, pretty much, what people were asking for. If it came down to it, we can always look up the database commands and figure out exactly what happened for serious issues (which we have done). While I would have liked to have worked on the problem until I came up with an ideal solution, my time was demanded on other projects and we had something that met the basic needs of the customers.
In the future we’ll hopefully get to revisit this project and use more accurate methods of logging line item activity. I would personally like to see it at the point where you can view exactly who added/remove/set/included/excluded what on a
given line item. We have a lot of issues that are higher priority right now, but the framework is in place to re-visit and complete this project.
So what did we learn here?
1. Developers are people too. We make mistakes and get overwhelmed just like anyone else. Patience from all parties involved is appreciated.
2. A lot of times there are no guidelines. These are real world problems that no class or textbook can prepare you for. More often than not it’s a few dozen of us trying to keep up with the demands and innovation of thousands of users.
3. Joel has a snazzy pocket protector.
4. Do not mess with the boss. They are not known for playing around. A few wise cracks in a blog entry will get you a lot of extra projects.
I hope this gave everyone some insight into how features are developed at RightMedia.
PS - As I was writing this, I discovered that the line item history barely works at all. At some point we changed something in the database and most of the functionality died. The funny thing is that no one noticed or said anything for at least two months. I guess the plus side of getting put on the low priority projects is that no one notices when you screw up.