So I used to joke around with Sapience, saying “We should totally do a QA dev diary!”
He would laugh, and I would laugh, and that was the end of it.
Then, months later, he took me up on it…

And now for something completely different
Actually, Sapience first went to my boss, who then approached me about writing a QA (Quality Assurance team) dev diary for Book 3, Volume I. My boss and I discussed it together and came to a conclusion: this is really hard! Our job isn’t the most exciting thing in the world, contrary to popular opinion. We each tried writing about QA structure and process, to get everyone acquainted with how we do things, but reading it was so incredibly dull that we scrapped it. QA doesn’t “play” LOTRO all day; it’s more like bending the game to our will. We fly around Middle-earth like some kind of techno-Valar, hunting down the bug-servants of Melkor before the player-Children of Iluvatar awaken.

On second thought, I guess that does sound like playing. Well, we can’t just do whatever we want; there is proper procedure and documentation to follow. My team (we are called “Team Ramrod”) creates a lot of that documentation. We usually end up “playing” with spreadsheets more often than anything else. Team Ramrod writes up the test plans that the rest of the QA teams use to check all of the new content in the game. To get this done, we are assigned to work closely with designers on their specific deliverables (new features, updates, bug fixes, etc.) for upcoming releases. This allows us to know exactly how we should approach the testing phase before any of it is implemented, and it lets us know ahead of time how much testing bandwidth (fancy QA-speak for time) a certain deliverable will need. Team Ramrod’s members are like the lobbyists of QA; we represent the QA team’s interests within the development process. The designers get the final say, but they rely on us to bring forth problems before they go live and balloon into an unmanageable mess.

Brynhildr vs. Orion
As for myself, during the past few updates I’ve been working closely with Orion on his player experience revamping crusade. I was there for his work in Ered Luin, Bree-land, and the Lone-lands, making sure anything Orion touched was getting the testing attention it deserved. We fought over a lot of little things, but we got the job done. In hindsight, arguing over a torch’s placement in the Retaking Weathertop instance was rather silly, but at the time it seemed important! I get riled up for the sake of the players sometimes, and Orion tries to improve things in the way he thinks is best. We don’t always see eye to eye, but things come together nicely when we’re done working with it.

In the coming update Orion got a bit ambitious and decided to make the entirety of the Volume 1 Epic Story more accessible to solo players and small fellowships. This is a noble goal and a well deserved addition; one that I’m very excited for. But that excitement is tempered by my QA experience: it’s a lot of stuff to change in one patch! And a lot of stuff to test! There’s only so much we’re able to get to in a reasonable timeframe, and labeling something as “solo-friendly” is kind of subjective. To prepare for the testing process I had to completely rewrite our old Volume 1 Epic Book testplans to ensure that Orion’s changes made the grade. I then had to approach one of the World Builders working on new travel routes—requested by Orion to help make travel between Epic Book locations more convenient—and get those properly tested, too. I hope you like horses and ponies to Meluinen, Gloin’s Camp, and Hrimbarg!

In addition, Orion got some new tech that allowed for the buff he created, Inspired Greatness, to be changed so it would scale based on your fellowship size. Now when under the effect of Inspired Greatness you will be buffed up according to how many group members you bring along. This was a drastic change to how it was previously used in our Lone-lands revamp and it required a lot of scrutiny from we testers. The new tech also allowed him to cut down on the number of changes he needed to make to get Volume 1 into the state he wanted. But then he decided to take that further and allow this buff into non-instanced areas where the Epic books take place, under heavy constraints. This was a potential QA nightmare! Players buffed to elite status while out in the world!? Cats and dogs living together!? Mass hysteria! We had to test extensively against potential exploits and ensure that it was indeed very limited in nature. We also had to look at all of the Volume One content from the perspective of many different class and fellowship compositions, while also considering the varied play styles of all players who want a chance to see the Epic Books for themselves.

Then there is the challenge of guiding the rest of the QA teams in the proper ways of testing the changes. I made initial exploratory passes while creating the testing plan, and then relied on other teams (“Team Awesome,” with preliminary help from “Team Chaos,” and some later grouping work by “Team Lakehouse”) to go through what I wrote and submit their own bugs on any issues they found. I had to create character template files to allow better accessibility to the content I wanted tested and quickly get the qualitative runs going (digging out proper equipment for each class takes a while). I then rallied the private beta testers and asked for their feedback on the changes before we moved into a later testing phase. This process is still ongoing and we’re currently rounding up any pressing issues and getting them taken care of before Volume 3 Book 1 is pushed to the live servers. Our public test server, Bullroarer, is the last net we have for finding edge-case issues that may have slipped by on our QA server environment and during the bug escalation process.

As you can see, there’s a lot of work involved in preparing a feature for release! I hope you all appreciate this little peek behind the curtain. On patch day, we QA testers are usually closing our eyes and waiting for the sound of explosions and screams of terror. If nothing bad happens, we can let out a short sigh of relief before starting up the process anew. The developers take us on a new ride every time, and we never quite know what exactly to expect!

Bio
Amanda "Brynhildr" Cosmos has been in the LOTRO QA department for over a year and a half. She enjoys Tolkien lore, harmonious quest flow, and playing LOTRO as a female Elf Champion.