Do you know anything about Bugzilla? Sometimes the answer to a simple, unexpected question like this can change our direction. Max Kanat-Alexander, our guest this week in episode 285, once answered this question with a resounding yes because of something he’d done earlier.

Max spent time triaging bugs for the Mozilla project while in college and had learned how to use Bugzilla. He would later deploy and administer the software for his employer, make the decision to write code that added support for PostgreSQL to Bugzilla, become the release manager for the project, and eventually become the chief architect and community lead.

This week you’ll hear how a young Max Kanat-Alexander decided to study computer science and what drew him to embrace the open source spirit through deeper commitment to the Bugzilla project over the course of his career. Max shares some great tips for those looking to contribute to an open source project and thoughts on where the technical lead sits in software engineering. Listen closely to understand the scope of problems people at each software engineering job level are tasked with solving.

Original Recording Date: 06-30-2024

Topics – Meet Max Kanat-Alexander, Experience with FileMaker, The Open Source Spirit, Upgrading Bugzilla and a Decision Point, Becoming a Release Manager, The Role of a Community Lead, Software Engineering Job Levels and the Technical Lead

2:10 – Meet Max Kanat-Alexander

• Max Kanat-Alexander is a Principal Staff Software Engineer at LinkedIn and is one of the technical leads for the developer productivity and happiness team.


• Max has been interested in computers since he was very young.
  
  
  
  • One of the schools Max attended had an old Tandy computer and a 286 computer.
  
  
  • When he found out one of these machines didn’t work. Max took it as a personal challenge to get it working again. Max tells us he learned a lot in the process.
  
  
  • Max’s family eventually got a computer, which he later broke.
  
  
  • Both before and during high school Max worked in IT technician roles.
  
  
  
  


• Max planned to major in theater, physics, or computer science in college.
  
  
  
  • When he began attending UC Santa Cruz, Max majored in theater.
  
  
  • “I quickly discovered that, at least at UC Santa Cruz, the theater major was very theater heavy, and I did not want to study the theory of theater. I wanted to act and direct and produce shows. And so I changed to my next equal interest… And the only computer related major that they had was computer science.” – Max Kanat-Alexander, on pursuing computer science
  
  
  • The computer science program was focused on learning to write code and programming languages.
  
  
  • In math class in high school, Max would write prime solver programs on his TI-83 calculator instead of paying attention.
  
  
  
  


• Max was in college from 1999 to 2003 and mentions there wasn’t a lot of mentorship out there to guide hopeful software engineers on career choices.
  
  
  
  • Max tells the story of a classmate in college who was hired to work for Sun Microsystems before finishing school, but that wasn’t something that happened often.
  
  
  • Max pursued comp science because he liked computers. Many of his classmates were either pushed to do computer science because it looked like math or science or they really had a passion for computers.
  
  
  
  

6:37 – Experience with FileMaker

• Max had been building FileMaker databases for people since his early teens. That first project was creating a database of stunt actors for theme park stunt shows.
  
  
  
  • Multiple FileMaker projects followed this, and Max created a FileMaker database for the university housing department, where he worked until the end of his college tenure.
  
  
  • FileMaker gives you a scripting language to work with and is what Max calls a nice introduction to the concepts of programming.
  
  
  
  


• “When I left school I still had more IT experience than I had programming experience, and really all these jobs that I’d had had been IT jobs where I just had coincidentally been making a FileMaker database or one time I programmed in Cold Fusion.” – Max Kanat-Alexander
  
  
  
  • Max eventually learned how to build basic webpages, but most of his experience was in IT jobs.
  
  
  • Max got really into Linux in college and installed Red Hat version 9 on a computer.
  
  
  • “The way that I am is I decided that I wanted to live on the edge and that I wanted to just constantly update to whatever the next development version of Red Hat was. And it turned out that the difference between Red Hat 9 and Red Hat 10 was there was no Red Hat 10. There was Fedora 1, and as a result I was one of the only people in the world who had been using Fedora effectively for months before it came out. And so I wrote the very first support web page ever for Fedora…. I was getting 120,00 unique visitors a month to a one page website.” – Max Kanat-Alexander
  
  
  • Max was hired as the lead technician for tech support in the Americas at Kerio, a company most famous for making a Windows Firewall back when Windows did not have a good built in firewall. This company made a mail server and a software router that ran on Windows.
    
    
    
    • Max answered phone calls and worked with a couple of other technicians to solve all kinds of interesting problems.
    
    
    
    
  
  
  
  

10:10 – The Open Source Spirit

• One day Max was asked if he knew anything about Bugzilla. The answer was a resounding yes.
  
  
  
  • “One of the ways I had spent my free time in college was triaging bugs for the Mozilla project. I just found this very interesting.” – Max Kanat-Alexander
  
  
  • Triage means someone has reported a bug, but no one knows if it is valid, if it can be reproduced, if it has sufficient information, or if it might be a duplicate of an existing bug.
  
  
  • At the time, all the triage work for Mozilla project bugs was done by volunteers, including Max. He had used Bugzilla extensively because it was the bug tracking system for the Mozilla project.
  
  
  
  


• Max’s company wanted him to install Bugzilla.
  
  
  
  • Bugzilla ran on MySQL as its database technology, but Max was much more familiar with PostgreSQL.
  
  
  • Max mentions for the programmer who really likes SQL, likes standards, and wants all possible features, PostgreSQL is a great option.
  
  
  
  


• Max used a fork of Bugzilla that Red Hat had made running on PostgreSQL, which was great until Red Hat stopped maintaining the fork, and there was not a way to upgrade the version as a result.
  
  
  
  • Max saw two ways to make the upgrade happen – copy the data from PostgreSQL to MySQL or make upstream Bugzilla support PostgreSQL. The latter was the choice Max made.
  
  
  • “I was very much keen on the open source spirit. And I felt like that spirit was ‘contribute back to the world rather than just back to yourself.’” – Max Kanat-Alexander, on his decision to make upstream Bugzilla support PostgreSQL
  
  
  
  


• Did someone model this idea of giving back for the greater good for Max earlier in his life?
  
  
  
  • Max triaged bugs because he was not a programmer. But he did it because he loved working with the technology and wanted to work more with it (bug triage being the only option). He loved computers, appreciated Mozilla, and he wanted it to succeed. It was an inherent motivation.
  
  
  • With the Fedora FAQ site, Max had encouraged others to write it multiple times before taking it upon himself to do so. He had participated in the IRC support channel and continued to answer the same questions over and over again.
    
    
    
    • “Anybody in that channel could have just done it, and it would have been wildly successful. After I while I started to get this impression that if I didn’t do something, nobody else was going to do it. That was very true on the Bugzilla project as well.” – Max Kanat-Alexander
    
    
    • If Max had not decided to write the support for PostgreSQL into Bugzilla, he is not sure anyone else would have added multi-database support.
    
    
    
    
  
  
  
  


• Can someone report bugs on GitHub today as a way to slide closer to development if they are not a programmer today?
  
  
  
  • Max would encourage anyone who wants to become a better programmer to work on open source. It is an environment where most of the time people really want your contributions. One of the hardest things in open source is getting others to do work for you.
  
  
  • Some non-programming skills may be even more valuable than programming skills in open source. If you have design skills, for example, open source projects would love to have you.
  
  
  • “There are no open source user interface designers. They do not exist.” – Max Kanat-Alexander
  
  
  • Max highlights contributions to open source as referenceable artifacts for your resume.
  
  
  • Max did over 10,000 code reviews for Bugzilla!
  
  
  
  


• How did Max find out about bug triage for Mozilla in the first place?
  
  
  
  • Max thinks he tried to file a bug and had to search for duplicates first. Once he noticed things were pretty messy he wanted to help clean them up.
  
  
  
  


• When Max decided to enhance the fork of Bugzilla, did he have an idea of how to do it or just confidence he could figure it out?
  
  
  
  • Max had enough programming experience to formulate opinions on the structure of software. He had written applications in Java and Cold Fusion but had to learn Perl to work on Bugzilla. Perl is easy to learn but one of the hardest languages to master according to Max.
  
  
  
  


• What advice does Max have for someone who wants to contribute to open source but is not sure what project to pick?
  
  
  
  • Max would advise finding something you use, something that matters to you, or something similar to what interests you because it will likely lead to being more involved. Go and see if you can contribute to a community in one of those areas.
  
  
  • Before contributing to an open source project, we can look at how frequently the commits are, how many issues are filed, and how responsive others are to the issues filed.
  
  
  • Look at the pull requests (or PRs) and see if there is a healthy discussion that happens as a result of them.
  
  
  • “You also will get a sense from looking at the PRs of the tone and the nature of the culture of the community and whether or not you want to be part of that. Some communities might be a little more aggressive and direct, and some communities might be a little more polite and reserved. And you have to decide which one more suits your style of work.” – Max Kanat-Alexander
  
  
  • Many times there is a link to a Discord or Slack channel that someone could check out. How alive is it?
  
  
  
  

20:03 – Upgrading Bugzilla and a Decision Point

• Max’s company wanted to upgrade Bugzilla to get new features, and he said he would make it happen, revealing later that he would be building multi-database support into the product. Others were a little skeptical of what Max intended to do, but as long as he finished all of his other work it was not a problem.
  
  
  
  • “At that time I actually worked more. I was at the office more than I had been in the past. I was spending a few hours a day working on Bugzilla in one way or another…. I would work on it…all the time because I just was so interested in the problem that I was solving.” – Max Kanat-Alexander
  
  
  • Nick says the choice to write code to upgrade Bugzilla aligns with Max’s desire to continue to work on the things that interested him, and this decision likely only strengthened his enjoyment of the project.
    
    
    
    • Max says “you feel like it’s your baby.”
    
    
    
    
  
  
  
  


• How was Max able to stay involved for so long?
  
  
  
  • Max made all of his changes in Bugzilla version 2.19 (which would become version 2.20 once released). The first number is the major version, and the second number is the minor version with even numbers being a stable version and odd numbers being a development version.
    
    
    
    • “You only want to be using the development release if you really want to live on the edge and you don’t want things to work. You’re fine with things breaking.” – Max Kanat-Alexander
    
    
    
    
  
  
  • Once Max made all the changes to version 2.19, his changes could not yet be released because version 2.18 would need to be released first.
    
    
    
    • Version 2.16 of Bugzilla had not been great, and a number of competitors showed up on the scene (like JIRA).
    
    
    • Max needed to be able to upgrade to version 2.20 to demonstrate that he had done work.
    
    
    • Just like back when he was using Fedora, he asked numerous people to get Bugzilla 2.18 released. But it wasn’t that simple as there were a number of bugs from 2.17 that needed to be resolved so 2.18 could be released. The release process involved other work too like packaging up the new version, sending out announcements, etc.
    
    
    • “So I was like, ‘fine. I’ll do it. I’ll just do it. I’ll just release 2.18.’ So, as one discovers as one goes through one’s history in life as a technician or any kind of technologist, if you accept responsibility for anything unpopular, it will become your job. Or if you express an opinion about anything that anybody else doesn’t want to work on, it will be come your job. This has happened consistently for the rest of my career…. So I became the release manager of the Bugzilla project.” – Max Kanat-Alexander
    
    
    
    
  
  
  • Max helped drive the release of 2.18, and once bugs were resolved in 2.19, 2.20 was released.
    
    
    
    • Through this process Max got more involved in the community and began to do code reviews for anyone wanting to commit changes to the project.
    
    
    • Max also had to ensure the test infrastructure was working for upcoming release testing.
    
    
    
    
  
  
  • Max really enjoys refactoring code and led the Bugzilla project through a number of refactoring efforts.
    
    
    
    • With Bugzilla, when a new feature was going to be released, it would get refactored. The refactoring process makes code easier to read.
    
    
    • When contributors built and submitted a patch, the Bugzilla team (including Max) would not accept the patch until it was refactored. But they would not stop there and would teach people how to properly refactor their code. Most of the time contributors appreciated the simplicity of refactoring and would eliminate even more bugs in their code in the process.
    
    
    
    
  
  
  • Max eventually became the chief architect and one of the co-leads of the Bugzilla project.
    
    
    
    • There was a lead named Dave Miller above Max and his co-lead of the project who would resolve any disputes between project leads when needed.
    
    
    
    
  
  
  • Max also became the community lead for the project and was in charge of recruiting new contributors, making the documentation better to onboard new contributors, improving the experience of existing contributors, encouraging people to get more involved, and making the community an overall more welcoming place.
  
  
  • Max went on to leave Kerio and start his own consulting company called Everything Solved (which primarily did Bugzilla consulting).
  
  
  
  

27:10 – Becoming a Release Manager

• It sounds like Max was perfect for the chief architect after seeing so many different aspects and elements of the project from bug triage to releases to writing and reviewing code to fix bugs. He knew the system as a whole very well as a result of that experience.
  
  
  
  • For purposes of defining terms, refactoring means cleaning up code to make it more readable, to make it simpler, and even more efficient – without changing the behavior that users experience.
  
  
  • When Max became a co-lead, that is what we might call a maintainer in the GitHub world.
  
  
  
  


• In taking on the role of release manager, Max knew nothing about the process. Dave Miller, one of Mozilla’s only systems administrators, helped him understand it.
  
  
  
  • This was around the time the Mozilla foundation started. Max learned entirely by asking Dave questions and eventually wrote down the release procedure he had learned.
  
  
  • Could someone learn release management without knowing how to code well or being a programmer? Could they sharpen their programming skills after starting to work in release management?
    
    
    
    • Max says it depends on the complexity of the release pipelines and systems.
    
    
    • Much of the role of release manager is project management – gathering the list of bugs that need to be resolved for a release to happen and checking in on timelines. The project management aspect can be done without having programming experience, but in most open source projects, you likely need to be a programmer to handle release management.
    
    
    • Max says the type of coding you have to do as a release manager is not that difficult. Mostly this would involve writing scripts to automate things that are normally done manually (but those things could still be done manually if needed). You do need to be competent with computers and handling people to do work at a minimum. The programming parts are quite learnable. Thinking about it again, Max says this is a pretty good entry point into programming.
    
    
    
    
  
  
  • Is release management a role or a function someone in another job takes on within most organizations today?
    
    
    
    • Max does not see it much as a dedicated role these days. The trend is to make software engineers do the release management themselves to understand the process and pain they have potential to cause in that process.
    
    
    • In cases of very complicated release systems, a dedicated release manager is needed. But in Max’s experience this is becoming more rare.
    
    
    
    
  
  
  
  

31:20 – The Role of a Community Lead

• How was the role of the community lead different from what Max had done previously?
  
  
  
  • This requires a broader level of responsibility. People start as being merely members of the community. If there’s something going on that you don’t like in the community or the documentation is not clear, it isn’t really your problem. That’s the community lead’s problem. You as a contributor / member of the community can go speak with other more involved members of the community for help or to get clarity.
  
  
  • The most important thing a community manager oversees is the onramp process for the project. This would be things like understanding where contributions are accepted, how to submit changes and the overall change process, expectations, specifying an easy entry point for newcomers, etc.
    
    
    
    • For example, Max came up with a system for Bugzilla to put issues into a specific queue for contributions by newcomers to the project. He as the community lead had to put in the work to create this system. These were relatively straightforward issues maintainers would never be able to handle but that could easily be handled by others who cared.
    
    
    
    
  
  
  • As soon as someone shows up to help, you must be very appreciative if you want to keep them as a contributor.
    
    
    
    • “People in open source show up because they love programming and they care about your project and they want to work with you. And they may have day jobs as programmers that they don’t like. And there’s various reasons that they don’t like them. One of them is they might feel like the thing that they work on is something that they don’t care about and that nobody appreciates. Very often, the pay that you get for work on open source is appreciation. In your life…the value that you get is both the sense of having delivered impact to the world and owning something really important for the world and then also appreciation. And then on the flip side, any negativity that you get, you don’t have to put up with that. Nobody’s paying you. You just leave.” – Max Kanat-Alexander, on the mindset of open source contributors
    
    
    
    
  
  
  
  


• Should open source contributors look for a project written in a language they are familiar with or one in a language they would like to learn?
  
  
  
  • Early in your career as a programmer, pick a project in a language you know well. Remember you will also have to learn a lot about the project and its code. Learning that and a new programming language at the same time is not something Max recommends.
  
  
  
  

35:02 – Software Engineering Job Levels and the Technical Lead

• What is a chief architect in software development, and what does that function do?
  
  
  
  • This role presents itself in different ways across different sizes of organizations.
  
  
  • Max’s job as chief architect in the Bugzilla project was to decide on the code architecture of the system. He would say it’s more like a technical lead role in many other companies today.
    
    
    
    • Someone in this role makes the final systems design decisions. This person is in charge of maintaining consistency, simplicity, and clarity as the system grows.
    
    
    • They should also be open to discussions with others who have a technical point to make.
    
    
    
    
  
  
  
  


• We’ve spoken to other guests who were technical leads in IT operations, for example. Are technical leads in the software development world usually individual contributors?
  
  
  
  • There are different types of tech leads according to Max. We will use the names Max has heard them called most frequently in the industry.
  
  
  • At the bottom of the technical hierarchy, we have software engineers and senior software engineers as common entry points. Think of the software engineer as someone who can write code but without enough experience to do it completely on their own, often times lacking an understanding of the best practices of systems and programming languages.
  
  
  • A senior software engineer (the next level up) can be trusted to write changes on their own. We might call this owning a feature from end to end and delivering it. You can give them work and not be as concerned about them doing it wrong.
  
  
  • The next level is staff and the lowest level that would really be considered a tech lead.
    
    
    
    • Tech leads have different duties depending on the team and the nature of the team and how many reports the manager of the team has. If the team is large, you would have some managerial type duties.
    
    
    • Tech leads set the architecture of the system and ensuring its quality, decide on the overall design, and make major technical decisions about datastores and languages and frameworks.
    
    
    • “They are helping guide the team towards improving their craft of software engineering, and they are also a connective tissue between their teams and other teams at the company. They are the technical connective tissue.” – Max Kanat-Alexander, on the responsibilities of a tech lead
    
    
    • A staff engineer may be acting as tech lead for more than one team (could be 1 -3 teams).
    
    
    
    
  
  
  • The next level is senior staff, and someone in this role owns multiple staff level projects. The nature of the role starts to change at this level.
    
    
    
    • They usually still write code and review it but are doing a lot more strategic work. A senior staff engineer may be leading 20-30 engineers or a project of immense scope and complexity. They could be coordinating with many teams across the company.
    
    
    • This person may own an entire problem like internal search for the entire company. The senior staff engineer would be responsible for writing the entire project plan for work that is going to take 6 months to 2 years. It would include how they will coordinate multiple teams to complete the work. This person would also be the central reviewer and approver of all technical decisions surrounding a particular system.
    
    
    
    
  
  
  • The next level up is principal, and someone in this role is getting farther away from coding on a daily basis.
    
    
    
    • Some principals still write code, but it is less common to be coding frequently.
    
    
    • At this level you work strategically and can be thought of owning a problem for the whole company.
    
    
    • Max is a principal at LinkedIn, and in the past he has owned problems like engineering metrics for the company. His only instruction for that project was “go make engineering more data driven.” It was up to Max to coordinate the work of 30-100 engineers to solve that problem. He would work on the strategy, the plan, and still be a technical reviewer for many things.
    
    
    • “It’s still my job to cause software to be built…. My job is…we have a technical vision. We need to get that technical vision delivered. And my job is to do literally anything required at the company to get that technical vision delivered. At my level there is no excuse that I cannot do something. I must be able to do anything that is required, any role that is required in order to accomplish whatever goal has been set for me.” – Max Kanat-Alexander
    
    
    
    
  
  
  • Above principal is the distinguished engineer.
    
    
    
    • This person essentially owns a sizable portion of the company. They may set a vision for hundreds of engineers within an organization across the entire company or sets a vision the entire company works toward.
    
    
    
    
  
  
  
  


• Nick says it seems like as your job level goes up, the scope is wider and the problem is more ambiguous when given to you. But you have more control over how the problem is solved and how the problem statement gets refined.
  
  
  
  • Max is involved in promoting people from staff to senior staff. While technical acumen and feedback from others are considered, the first thing discussed is scope and ambiguity.
  
  
  • For promotion purposes, do companies want you to have a wide impact both inside and outside the company / in the greater industry?
    
    
    
    • This depends on the level you’re at and the company’s expectations, but Max says you’re usually not expected to have a broad impact on the world until you reach principal or above.
    
    
    • “At least as an engineer, what does matter is impact of the work that you’ve done.” – Max Kanat-Alexander
    
    
    • If you work on the product your company makes, you do need to have an impact on the world outside your company.
    
    
    • At a lower level you have this impact by writing the code that fixes a product bug or produces a new feature, etc. The impact is felt upon product release.
    
    
    
    
  
  
  
  

Mentioned in the Outro

• Remember that an open source project is a technical community. Max shared some great ways we can evaluate one the same way we would an online forum or other community group.


• Can we describe the increase in the scope of problems we’re trying to solve in our careers? It’s a great story to share in interviews!
  
  
  
  • The idea of an increase in problem scope extends to any technical career (not just software engineering).
  
  
  • The scope of the problems you’re solving is not increasing, maybe you need to learn a new skill to make that happen.
    
    
    
    • If you have administered a system but never designed and deployed it, moving into a consulting role focused on this system’s use across many customer environments can get you more experience as an option.
    
    
    • Get to know your technical leads, and figure out how you can help them. Maybe part of what they need to do can be delegated to you.
    
    
    • Could you shadow the technical lead or ask to be part of some of the planning sessions they have to lead? Learning from the technical lead can help you build relatable skills to take on that role someday.
    
    
    
    
  
  
  
  


• Here are a couple of recommendations for more detail on the technical lead / tech lead role:
  
  
  
  • Episode 278 – Uncovering Empathy: The Greatest Skill of an Inclusive Leader with Marni Coffey (1/3) – Marni Coffey spoke about progressing into a technical lead role in IT Operations
  
  
  • Episode 246 – Learning to Lead and a Culture of Excellence with Dr. Sirisha Kuchimanchi (2/3) – Sirisha discusses being a technical lead / team lead in this episode
  
  
  
  

Contact the Hosts

• The hosts of Nerd Journey are John White and Nick Korte.
  
  
  
  • E-mail: [email protected]
  
  
  • DM us on Twitter/X @NerdJourney
  
  
  • Connect with John on LinkedIn or DM him on Twitter/X @vJourneyman
  
  
  • Connect with Nick on LinkedIn or DM him on Twitter/X @NetworkNerd_