Resume

Hello!

Hello, my name is Mark and I work at Fastly as a Staff Software Engineer.

NOTE: Yes. That is my wife and kids. But I’ve no idea who the big blue guy is 😉

Previously

• Staff Software Engineer at BuzzFeed
• Principal Engineer at BBCNews
• Principal Engineer at StormCreative

Working Together / Testimonials

The following quotes (all very kind) are from people I’ve worked with. Everything quoted below was unsolicited and unexpected feedback, which is just so lovely! I’ve never asked anyone to “please write something nice about me”. All of these wonderful comments were genuine and truly appreciated.

Summary

I ideally want to get across three fundamental aspects about me:

1. I care. I care about the people I work with, the organisation I work for, and our customers.
2. I’m passionate about programming and the openness of the web
3. I love getting the chance to learn and experience new technology

Connect

You can find me online at the following locations:

• integralist.co.uk
• github.com/integralist
• twitter.com/integralist
• linkedin.com/mark-mcdonnell

Brief History

• Fastly (2020-present)
• BuzzFeed (2016-2020)
• BBC (2013-2016)
• Storm Creative (2001-2012)

Fastly (October 2020 - present)

BuzzFeed (June 2016 - October 2020)

BBC (Jan 2013 - June 2016)

Storm Creative (Feb 2001 - Dec 2012)

⚠️ From here on we get deep into details. It’s not necessary to read on unless you’re really interested in what I’ve been up to 🙂

Impact!

I’d like to share the projects I’ve worked on over the years and the impact/value this work has provided.

NOTE: My ‘impact’ has primarily been as a remote worker. My hope is that the following list demonstrates how I’ve made a positive impact (both as an IC and as a Team Lead) while also being a 100% remote worker with a family to support and balance against.

• Impact at Fastly
• Impact at BuzzFeed
• Impact at BBC

🏢 Fastly

2024

• What: I wrote a document “Terraform validation for API modifications” that proposed a solution for how we can validate API changes against Terraform to prevent breaking changes making their way into production.
  Why: There was an incident where a Fastly API introduced a breaking change that affected an application I was on-call for. It was a confusing bug and took a lot of time to investigate and identify and I was concerned for our customers and how they might be affected (especially considering how difficult Terraform can be to work with in general).
  Impact: This proposal document kick-started a new initiative to ensure Terraform is treated as a first-class citizen within the Fastly ecosystem.

• What: Redesigned internal boilerplate project used for create new Go services at Fastly.
  Why: As part of my work within a new team (although I was the sole engineer for quite a while) I needed to get a new API application onboarded with an internal platform that exposes the API to the public internet. The recommendation was to use an internal boilerplate project, which turned out to be quite stale, un-maintained, and in need of a redesign.
  Impact: Quite a few internal teams were onboarding onto the same platform around that time and were able to benefit from not only the improved tooling I built, the improved resiliency code I wrote, but also the in-depth documentation I wrote to provide context and understanding around how the platform itself functioned. Ultimately making it much easier for developers to onboard their Go applications.

• What: I created a public tool for validating Fastly’s Compute cache semantics (code).
  Why: Customers were having trouble understanding how HTTP cache semantics worked with the Fastly Compute platform (which at the time of writing doesn’t have a HTTP cache API like Fastly’s VCL product offering).
  Impact: Customers are able to validate and understand how caching works with Compute. They can define a set of HTTP calls and see how they are cached, and can assert against the cacheability of the calls. This work also helped to identify a bug in the underlying HTTP caching implementation.

• What: I implemented a complete CI/CD pipeline using GitHub Actions, along with dynamic service creation for multiple environments using Terraform Cloud.
  Why: We wanted to move towards Iac (Infrastructure as Code) and automated releases.
  Impact: We’re able to easily spin up environment specific instances of our service(s) and all the benefits of Continuous Deployment.

• What: I moved into a new team working exclusively on Domainr.
  Why: It was felt my skill set would be ideal to help not only take over the project as the lead developer but to begin the planning, architecture and development on new features and products based around Domainr and its subject area.
  Impact: Too early to say. But one month in and I’m very excited about the future.

• What: Implemented a solution to enable public repo forks to have access to GitHub Actions secrets.
  Why: Fork PRs do not have access to GitHub Action secrets which makes validating the correctness of community contributed code much harder.
  Impact: An authorised user (i.e. a contributor with WRITE access to the repo) can now comment /ok-to-test sha=<SHA> and this will trigger the CI to run with the correct secret(s) access.

• What: Built an interactive CLI prompt to generate OpenAPI schemas.
  Why: Two reasons: the first was that developers were unfamiliar with OpenAPI and unsure how to write the schemas (the learning curve wasn’t steep but substantial for busy devs), and second: we use our schemas for auto-generating API clients and so it was easy for developers to design schemas that would break the code-generation process or produce invalid client code.
  Impact: Consistency in our OpenAPI schema design, which helped to improve the generated client code interface (so a much better developer experience) as well as reduced errors in the API clients overall.

2023

• What: Implemented production ready Single-Sign On (SSO) implementation in Fastly CLI using PKCE (Proof Key for Code Exchange) OAuth flow (this was a follow-up to an earlier POC which was completely rewritten along with a project wide refactor to better support the integration).
  Why: Part of a larger initiative to upgrade our authentication and authorization model, so that we can offer more features to customers.
  Impact: Customers will get more flexible and advanced sign-in features.

• What: Implemented Wasm binary metadata annotations (i.e. data collection).
  Why: Fastly and their customers have a mutual goal of attaining better visibility into what services have been deployed in production.
  Impact: Customers gain insights to dimensionality such as by industry vertical and by scale (e.g. what Rust packages are Compute customers in the commerce industry with over 10k requests/sec using?). They also benefit from improved security vulnerability detection (e.g. Fastly is able to recommend package version upgrades).

• What: Asked to lead the internal ‘API Versioning’ Working Guild.
  Why: Lack of direction meant we weren’t making as much progress as the organisation would like.
  Impact: I researched, documented and presented a slide deck that architected a complete (green field) solution for an automated API versioning system that used satellite systems to run a scheduled release cycle with a pull model to acquire individual API application metadata, which inturn would publish a new external consumer API interface version at the appropriate time (yeah, that was a mouthful I know 😄).

• What: Moved into a new “Customer Developer Tools” team, responsible for managing, developing and maintaining/supporting the various tools Fastly customers use to interact with our platform.
  Why: My responsibilities were growing beyond the boundaries of the Developer Relations team.
  Impact: I was able to focus more of my time and energy on supporting these critical customer developer tools.

• What: Invited to join internal API Working Guild.
  Why: I’m the core maintainer for Fastly’s Terraform provider(s), CLI and multiple API clients, so my insight and expertise in those areas were important for ensuring the quality of our new API design.
  Impact: I’ve been able to help guide the design of the API to account for specialised consumer cases that otherwise would have been neglected.

• What: Rewrote Fastly’s (very complex) Terraform provider from scratch (https://github.com/Integralist/terraform-provider-fastly-framework). Why: New HashiCorp framework was incompatible with their previous SDK and so it was important we started migrating before HashiCorp published a deprecation notice.
  Impact: Customers will benefit from a more modern provider implementation and a whole bunch of long standing bugs resolved.

2022

• What: Implemented a proof-of-concept PKCE (Proof Key for Code Exchange) OAuth flow for the Fastly CLI (example gist).
  Why: Part of a larger initiative to upgrade our authentication and authorization model, so that we can offer more features to customers.
  Impact: Customers will get more flexible and advanced sign-in features.

• What: Implemented a new code-generated Go API client with the help of openapi-generator.tech.
  Why: Part of a larger initiative to use the OpenAPI specification to code-generate language API clients.
  Impact: We have a lot of customers interested in Go who are now able to interact with our platform.

• What: I was the SME (Subject Matter Expert) on a cross-team collaboration for supporting custom headers in healthchecks for Terraform.
  Why: An important customer required this feature as part of a large-scale event.
  Impact: We completed the project ahead of schedule and the event was a success.

• What: Implemented a solution to reduce API calls in the Fastly Terraform provider (PR).
  Why: The provider was making unnecessary API calls that was causing network and performance problems, and was affecting our platform reliability.
  Impact: I reduced the time for a ‘plan’ (and the time it takes to run the integration tests) by ~70%! This improves the quality of life for our customers, while also helping to protect the Fastly API, and benefits the developers working on the provider.

• What: Implemented a new code-generated Rust API client with the help of openapi-generator.tech.
  Why: Part of a larger initiative to use the OpenAPI specification to code-generate language API clients.
  Impact: We have a lot of customers interested in Rust who are now able to interact with our platform.

• What: Implemented retry behaviour for critical section of customer facing platform software.
  Why: A hard to reproduce, and intermittent, network failure was causing services to restart at the infrastructure layer, which was a slow and costly process.
  Impact: Developers wouldn’t waste cycles having to manually re-start flaky/failing CI jobs, and a large reduction in service remediation latency.

• What: Designed a PKCE (Proof Key for Code Exchange) OAuth flow for existing authentication systems.
  Why: We wanted all our public clients to be able to support OAuth.
  Impact: We increase the security and flexibility of our authentication systems.

• What: Designed and developed a CLI based strategy (in go) for integrating an OAuth user flow. This included co-authoring the relevant Rust work (in a separate internal auth service) to support different Fastly clients (i.e. a client might be one of Fastly’s existing web properties like developer.fastly.com or the Fastly CLI).
  Why: Users of the Fastly CLI could not configure more than one token. This would mean the token typically, for convenience, would be a long-lived token (e.g. no expiry). This was a tedious flow for users as they needed to manually interact with different areas of the Fastly UI before then having to go back to the CLI to apply a token.
  Impact: Users have a slick and easy authentication flow that is more secure and provides better best practices (i.e. using short-lived tokens that regenerate frequently).

• What: Developed features and bug fixes for, as well as managed and coordinated, a major v1.0.0 release of the Fastly Terraform provider.
  Why: The provider had been at 0.x for the past couple years and more generally was not in a great state.
  Impact: A greatly simplified user experience (UX) via a more consistent interface, the fixing of multitude bugs, and resolving a potential issue that otherwise could trigger data loss, resulted in a much improved product for Fastly customers.

“This is a HUGE milestone for the provider, it’s come a long way since we adopted it and we’ve always been building up to being able to cut this v1 release.

Congrats to @integralist for his hard work, dedication, and commitment to navigate us through this project.”

– Principal Engineer @Fastly

2021

• What: Migrated from Travis CI to GitHub Actions, which included general refactoring of scripts, implementing reusable jobs and figuring out how to efficiently persist data between jobs (I also added lots of internal documentation).
  Why: To reduce complexity in our CI/CD pipelines.
  Impact: Our CI is much easier to understand, maintain and contribute to for internal staff (I also wrote about GitHub Actions as a side-effect).

• What: Published a best practices guide for writing Terraform using the Fastly Terraform provider.
  Why: We noticed an increase in customer support queries related to Terraform and it being a complex topic to understand.
  Impact: Customers reported an appreciation for the official guidance as it helped to elucidate an otherwise confusing combination of tools and concepts.

• What: Acted as an ambassador for Fastly support on Stack Overflow (Fastly Stack Overflow profile).
  Why: To help the community of Fastly users.
  Impact: We unblocked customers and enabled them to become productive, as well as generally feeling supported.

• What: I designed, wrote and coordinated the release of a Terraform training module that also focuses on integration with Fastly.
  Why: We had noticed an ever growing increase in interest with Terraform.
  Impact: Our customers can feel confident and supported when thinking of managing their services using Terraform.

• What: Wrote a custom golang program to parse a bunch of OpenAPI schemas and to auto-generate a golang based API client.
  Why: To identify potential problems with our OpenAPI schemas and whether we can offer custom generated API clients to our customers.
  Impact: We have a viable solution that will ensure our API endpoints and API clients stay in-sync with minimal effort/cost.

• What: Defined a new retro process.
  Why: There was none defined.
  Impact: We now have effective and valuable retros 🎉.

2020

• What: Reaching out to our customers (and potential customers).
  Why: To help answer complex questions related to the fastly platform.
  Impact: The fastly organisation is more approachable and better understood from a technical perspective.

• What: Coordinated wide net of user feedback (both internally and externally).
  Why: To help drive decisions around which API client libraries we should maintain (or which new languages to support).
  Impact: We have a clearer understanding of where we should focus our resources.

🏢 BuzzFeed

2020

• What: worked across different teams promoting certain patterns to help reduce our logs (e.g. sampling 2xx’s & single-log context middleware for http servers).
  Why: we had started to exceed our cost agreements with our metric/log aggregation provider.
  Impact: managed to reach our log production goal, while having the opportunity to work and share ideas with lots of different teams across the org.

• What: I was the sole engineer designing, communicating (via RFC process) and building a global rate limiting feature (read my post about it) that integrated at different layers of our infrastructure (CDN, Routing + a new ‘rate control’ service).
  Why: teams across the organization were struggling to handle rate limiting within their services and Core Infrastructure (the team I work in) found through user feedback that this would be a valuable task to take on.
  Impact: I was able to release a feature that provided our organization with DoS level protection at ‘the edge’ (e.g. reduced load on our internal infrastructure) while decoupling rate limiting logic from our critical path, along with granular client identification context to be more useful than a simple IP deny list.

• What: spent three weeks tackling problem of metric production across ~600 microservices.
  Why: we had run over our allotted threshold Impact: we saved the company ~$40k! We also educated ourselves greatly on how our provider determines ‘custom metrics’ and how to better use distribution metrics over standard histograms.

• What: wrote Python script that parsed any metrics produced by our ~600 microservices and cross-referenced them against our ~1000 dashboards and ~1000 monitors to identify which metrics were actually used.
  Why: in the face of the Covid-19 pandemic we needed to reduce our metric costs.
  Impact: teams were able to run the script to efficiently identify unused metrics, and it provided direct links to the relevant monitors and dashboards (and even the specific graphs within those dashboards).

• What: Implemented ‘graceful shutdown’ within shared libraries for HTTP servers written in Go/Python.
  Why: EC2 updates (for servers we manage under ECS) were causing containerized services to fail.
  Impact: The use of graceful shutdowns meant in-flight requests had time to complete once receiving a SIGTERM to the service by EC2.

2019

• What: Created an RFC for an updated Disaster Recovery plan.
  Why: Our ‘Plan Z’ (static rendering) approach wasn’t clear to some teams/influencers.
  Impact: Summarized our current disaster recovery approach (i.e. in most cases relying on serve-stale) and highlighted the ways we were building confidence in our Plan Z approach by smoke testing the behaviours on a regular basis. As well as documenting possible active:active CDN approaches where an alternative provider supports a programmatic edge comparable to Fastly.

• What: Refactored and rewrote our static rendering ‘disaster recovery’ edge logic/process.
  Why: Modifications to our VCL for serving custom error pages from the edge had broken legacy DR logic that would attempt to pull error pages from AWS S3.
  Impact: Improved maintainability, understanding, effectiveness, resilience.

• What: Consolidated different debug headers and query parameters.
  Why: As the list of services along a request path increased (we introduced different proxies) the number of debug entry points increased.
  Impact: Reduced ~4 different debug entry points to 1, allowing engineers to more efficiently access the information they needed.

• What: Implemented new endpoint that directs traffic to a custom instance of httpbin.
  Why: To enable engineers to test specific scenarios and behaviours of different aspects of our system.
  Impact: Teams are able to debug how different proxies will affect the request flow, and can now test different areas of our CDN logic (such as verifying if we’re serving stale when expected).

• What: Defined service contracts (example).
  Why: Complex services (such as in technical areas teams were unfamiliar with) could behave like black boxes.
  Impact: Provided service consumers with a clear definition of expectations, enabling teams to better understand what a service would do for them.

• What: Wrote a guide to HTTP caching.
  Why: As engineers we have a responsibility to understand the platform we write software for.
  Impact: Shared understanding of how to properly use HTTP caching directives.

• What: Extended OpsBot Slackbot application with /oncall command and generate reports that are dynamically added to incident document in Google Drive.
  Why: Seemed like a useful feature to add as part of HackWeek.
  Impact: People ‘on-call’ can now track what they work on and generate a report which becomes a permanent record.

• What: Built (go based) CDN smoke test verification service.
  Why: Migration of behaviours from CDN to new Perimeter service was proving complicated.
  Impact: Able to verify existing behaviours, and deploy more confidently when the smoke tests continue to pass.

• What: Documented specific definitions for SLI, SLO, SLA.
  Why: To help people understand what these terms mean and why they are useful.
  Impact: Engaged more teams in defining SLA’s for their services along with their own internal SLI/SLO’s.

• What: Planned and designed project to migrate VCL logic from CDN layer to a new ‘perimeter’ service.
  Why: Reduce the amount of custom routing and behavioural logic within our CDN (which is a ‘black box’ for most engineers).
  Impact: Enabled BuzzFeed’s ability to failover to a different CDN provider, thus avoiding vendor lock-in and improving our CDN resilience.

• What: Implemented CLI command for authenticated purging of URLs in Python.
  Why: To enable BuzzFeed staff (inc. tech support) to purge a given URL.
  Impact: Reduced the number of ‘blessed’ users given direct access to our CDN control panel, thus reducing the scope of potential user mistakes for a critical piece of our infrastructure.

• What: Locked down purging of individual CDN URLs to authentication by API key.
  Why: Avoid bad actors engineering a distributed attack of our cache nodes.
  Impact: Increased platform resilience to cache dropping exploits.

• What: Moved to ‘Core Infra’ (specifically the Infra Edge) team.
  Why: To increase the security and resilience of our Edge.
  Impact: Excitement from everyone in Core Infra to have me onboard. Seems my reputation preceded me ❤️

2018

• What: I replaced BuzzFeed’s use of NGINX+ (an expensive commercial product used as part of a critical piece of BuzzFeed’s infrastructure) with the open-source version.
  Why: This was a hack week project. It took one day to develop the changes, one day to test and verify behaviours in a staging environment, followed by a quick rollout to production.
  Impact: This saved the organization $60,000 a year in licensing fees.

• What: Designed and co-implemented new authentication system built in Python on top of AWS Cognito.
  Why: We needed to decommission a legacy authentication system coupled to a 10yr+ monolithic Perl application.
  Impact: Enabled more services to offer authentication, thus allowing more community driven features across our products.

• What: Built a Python package that wraps scrypt.
  Why: Provide a consistent interface when requiring a hashing function.
  Impact: Engineers unfamiliar with security protocols (e.g. hashing mechanisms or the difference between that and encryption) could use hashing without having to understand the implementation.

• What: Helped promote the benefits of Kim Scott’s ‘Radical Candor’, Marshall Rosenberg’s ‘Nonviolent Communication’ and Fred Kofman’s ‘Authentic Communication’ to different teams across BuzzFeed.
  Why: Effective, clear and compassionate communication benefits everyone.
  Impact: Teams were becoming more productive with the increased confidence to give the most appropriate and direct feedback necessary to catch both interpersonal issues and address team concerns.

• What: Introduced Wednesday lunch videos/presentations.
  Why: To motivate and inspire our development teams.
  Impact: People had fun listening to interesting topics (not all tech related), and having a source of conversation and discussion beyond the lunch hour and in some cases helped to inspire changes that fed back into the company.

• What: Designed and implemented a Python Tornado web handler decorator responsible for acquiring/caching/revalidating an asymmetrical public key.
  Why: To protect services from unauthenticated access (the public key used to sign JWTs provided by an authentication proxy we had built in Go).
  Impact: Helped engineers to integrate with our custom built authentication service and provide a consistent experience across the platform.

• What: Co-designed and co-implemented a Go based reverse proxy acting as an authentication layer infront of BuzzFeed services.
  Why: Part of a plan to decommission our legacy authentication system.
  Impact: The use of JWTs helped to develop a stateless system for providing authenticated access to services, thus making the system easier to reason about, and enabled teams to decouple themselves from our legacy Perl stack.

2017

• What: Implemented README validator in Python.
  Why: As part of BuzzFeed’s “Better Docs” initiative (of which I was a core member of its Working Group).
  Impact: This helped BuzzFeed to track the success of its new “Doc Day” event, which supports staff across the org in reviewing and improving software documentation.

• What: Led the effort to document, improve, and educate others on the state of BuzzFeed’s monitoring.
  Why: Our monitoring system was noisy, which made going on-call a much more stressful proposition.
  Impact: I also wrote a community blog post sharing and explaining a lot of what we did, along with sharing a template Runbook for operational safety.

• What: Core member of the BuzzFeed “Better-Docs” Working Group.
  Why: We aim to improve documentation and its discoverability for BuzzFeed Tech.
  Impact: We standardized the doc formats, the creation and maintenance of doc tooling, and continued to educate ourselves and the BF Tech community about the importance of good documentation.

• What: Tech Lead for the Site Infra ‘Resilience’ team.
  Why: Necessary to help improve the stablity and resilience of BuzzFeed’s existing services while helping to educate development teams on the best practices.
  Impact: We designed a disaster recovery strategy specific for BuzzFeed’s needs (called ‘Plan Z’) which helped document failure scenarios and failovers for our service providers (alongside that primary task we helped improve the resilience for BuzzFeed services).

• What: Built an operations Slackbot in Go.
  Why: I developed this as part of BuzzFeed’s ‘Hack Week’.
  Impact: Enabled all BuzzFeed staff (whether technical or not) to spin up either a public or private incident channel in Slack, while allowing interested parties to be auto-invited based upon an emoji reaction implementation. The tool also allowed people to search for operational runbooks stored within our organizations Google Drive.

• What: Designed and implemented a round-robin, multi-cloud provider nginx solution for serving static assets.
  Why: To help provide greater resilience when serving client-side static assets such as images or scripts.
  Impact: The tooling we built around this implementation helped to make the process of deploying and serving static assets efficiently much easier.

• What: Technical Lead and architect for a dynamic video player service.
  Why: To enable asynchronous editor workflows.
  Impact: Enabled flexible video selection for end users, while helping to promote BuzzFeed’s own brand of video content away from YouTube (which would otherwise require us to lose potential profit).

• What: Designed and implemented a Go CLI tool for deploying Fastly VCL changes.
  Why: The existing process for deploying Fastly VCL was manual and time consuming, and prone to mistakes.
  Impact: Helped unblock engineers who needed a more efficient way to rollout changes, while allowing them to diff and validate changes locally without having to sign-in to Fastly’s otherwise confusing UI.

• What: Refactored existing HTTP Cache Client Python package.
  Why: Original design was a facade around a multi-tiered cache abstraction over a Python HTTP client. This proved to be too limiting for engineers.
  Impact: Utilized an Adapter pattern internally to provide a unified interface, thus making it easier to provide custom HTTP clients instead of locking the caller down to a single client type.

• What: Implemented GitHub hook mechanism for detecting API changes and generating updated documentation.
  Why: Documentation would often go stale because engineers would make changes but not re-run the rendering tools to generate new docs.
  Impact: Enabled engineers to make changes without having to think about generating new documentation or having to know how to use the tools for generating documentation.

• What: Refactored legacy VCL code and spent time building necessary abstractions.
  Why: Original code was difficult to understand and meant a blessed set of engineers understood how it all worked.
  Impact: Opened up the CDN to more engineers and helped to provide abstractions (such as for logging) to make working with VCL easier for those new to the language.

2016

• What: Migrated Fastly’s version of varnish/vcl 2.x to standard 4.1.
  Why: Support switching to an alternative backup CDN.
  Impact: Strengthened our relationship with Site Reliability, while also building confidence in a failover CDN.

• What: Designed and implemented generic GitHub Pull Request template file.
  Why: Consistency and standardization of how to structure pull requests. We based the final format was on an old blog post I wrote (back before GitHub offered their template feature).
  Impact: Clearer problem/solution descriptions that enabled engineers not familiar with the services to understand the changes proposed.

• What: Implemented a smoke test scheduler service in Python.
  Why: Catch regressions with BuzzFeed’s primary routing service.
  Impact: Helped engineers to identify integration problems where routing changes would have adverse unexpected effects.

• What: Led development across a global team (primarily US based), and the rollout of a new critical routing service.
  Why: The routing behaviours for BuzzFeed was accessible to a set of engineers who understood the CDN and VCL.
  Impact: Enabled the entire engineering department to make routing changes based on complex sets of dynamic input and requirements via a simple config driven workflow.

• What: Porting of Perl services over to Python BFF services.
  Why: Decommission of 10yr+ monolithic Perl application.
  Impact: Increased BuzzFeed’s recruitment opportunities by expanding the amount of services written in Python (compared to hiring Perl developers), as well as improving the code quality of those services migrated.

• What: Proposed usage of specific Python linters and related tooling.
  Why: Code consistency and easier debugging of code.
  Impact: Improved code quality.

🏢 BBC

2016

• What: Defined “The Perfect Developer Qualities”.
  Why: To inspire and motivate my colleagues.
  Impact: Engineers from across the organization reached out to me to share their thoughts, feedback and general appreciation for the time and consideration (as well as the obvious past experience) that led to this ideal list of character traits.

• What: Released the open-source project go-elasticache.
  Why: Share useful tools that would benefit others.
  Impact: Improved the developer experience when working with AWS’s ElastiCache service.

• What: Led performance testing, analysis and resolution of scaling issues for the BBC’s internal “Mozart” platform (written in Ruby).
  Why: Network bottlenecks were causing issues during load testing.
  Impact: Helped to identify specific service within the architecture that resulted in it being rewritten in Go and thus resolving the scaling performance issues.

• What: Implemented simple, yet performant, URL monitoring system in Bash called Bash Watchtower.
  Why: Previous version was a complicated and over engineered Node application (a colleagues pet project, and no one in the organization used Node at the time). Being full of NPM packages made installing and running it a slow process.
  Impact: Improved deployment speed, helped other engineers understand the code base by using a language they were more familiar with, and simplified the code.

• What: Created and led BBC News “Coding and Architecture” working group.
  Why: We were responsible for teams following best practices.
  Impact: Improved the quality of new services developed, and helped us to communicate with a wider range of the organization.

• What: Co-designed and co-implemented the BBC News “Mozart” platform.
  Why: Simplify the ability to build up dynamic page composition.
  Impact: Enabled teams to build up complex pages of individual components. It also helped path the way for the organization to move away from internal hosted system to the AWS platform, while enabling developers to use easier languages and tools.

2015

• What: Represented BBC at AWS’ week long re:Invent technical conference in Las Vegas.
  Why: To learn more about the new AWS services that could benefit the organization.
  Impact: Networking with lots of different companies and helping to promote the work that the BBC does (specifically the engineering arm of the organization).

• What: Co-designed and co-implemented a Go based CLI tool called “Apollo”.
  Why: Abstract away certificate based authentication to internal APIs.
  Impact: Enabled teams to deploy services to the AWS platform.

• What: Team Lead for BBC News Frameworks team.
  Why: To help my team grow and to learn.
  Impact: Helped to promote a large segment of my team into senior position roles.

• What: Won “Connecting the News” Hack Day event.
  Why: Event for different news organizations to come together around a shared data source (provided by the BBC) and to see what interesting tools and services can enhance that data.
  Impact: Networking with engineering teams across different news platforms helped to inform potential ideas for our own services. Showcased BBC News as a great place to work.

• What: Released BBC Newsbeat v2.
  Why: First fully AWS product from BBC News.
  Impact: Started the movement of services from using an internal hosting platform onto the AWS platform.

• What: Tech Lead for General Elections.
  Why: The General Elections was a big event for BBC News.
  Impact: Successful build, deploy and monitoring of election reporting platform.

• What: Rebuilt and migrated BBC’s Market Data to AWS using the BBC’s open-source Alephant framework, of which I was a co-author.
  Why: Fix an old and un-maintained, yet critical, data service.
  Impact: Modernized and improved this essential financial market service for its stakeholders and enabled further extension by other engineering teams.

2014

• What: Designed and implemented “Jello” which was an internal synchronization service between Trello and Jira.
  Why: Teams preferred to use Trello, while the rest of the organization was using an old version of Jira.
  Impact: Teams benefited from the speed and feature set of Trello without having to manually track tasks back into Jira for the rest of the organizations visibility.

• What: Won “Most innovative use of Technology” BBC News Award (Docker CI).
  Why: Legacy Jenkins CI required a centralized operations team.
  Impact: Enabled teams to build and deploy software using any language or platform supported by Docker.

• What: Won “Best Public Relations of the Year” BBC News Award (Pro Vim).
  Why: I like writing and sharing information that helps people be more proficient with the tools they use.
  Impact: Book was well received and opened the Vim editor to wider range of engineers.

• What: Co-designed and co-implemented cloud based distributed load testing tool.
  Why: Existing solutions weren’t able to scale with our platform.
  Impact: Enabled engineers to load test their services at scale and identify performance bottlenecks.

• What: Organized public speaking event with Sandi Metz.
  Why: To build an engineering network event for the London tech community.
  Impact: London tech community got to see an otherwise often unseen internal look at BBC engineering talent and were able to discuss topics of interest.

2013

• What: Voted “Developer of the Year” at the BBC News awards.
  Why: I had made sure to reach out and affect in a positive way every single aspect of the business and to make a real difference to the developer community within the BBC.
  Impact: A genuine sense of pride that I was able to achieve what I set out to do: make a difference.

• What: Led development of the BBC News ‘responsive navigation’ redesign.
  Why: Part of the new BBC UX rebranding.
  Impact: Resulted in communication with product, design and engineering teams across the entire breadth of the BBC platform. Leading to a new responsive navigation that was able to accommodate all perspectives and requirements.

• What: Invited to speak at Mozilla offices in Paris.
  Why: To discuss the BBC News responsive images technique to browser vendors such as Apple, Microsoft, Opera, Mozilla and Google.
  Impact: I was able to establish myself as a person of interest to this organizations and an expert in the field when it came to client-side development.

• What: Implemented new BBC UX framework.
  Why: The BBC brand was undergoing a organization wide redesign.
  Impact: This was a long and deliberate implementation and rollout process that helps re-establish BBC News as a leader in the responsive mobile development space and helped showcase BBC News engineering talents.

• What: Implemented new BBC responsive images solution.
  Why: Scalable and responsive images was not widely supported by browsers with native APIs, requiring the development of custom solutions.
  Impact: Public BBC News post proposed our solution to the then difficult problem of how best to serve images in a scalable way to browsers and mobile devices.

• What: Introduced the use of GruntJS.
  Why: Using Ruby and Rake although engineers were unfamiliar with the language and were afraid to make changes or to build new tasks.
  Impact: Improved the ability of engineers to automate project tasks using JavaScript.

• What: Member of the BBC’s GEL Responsive Working Group.
  Why: To hear different engineering perspectives on how best to develop new UX designs.
  Impact: Simplified specific aspects of GELs design.

Talks

• Living on the Edge (Video): Presentation on our use of Fastly CDN and an internal service golang proxy service I built called “Perimeter”.

• Site Router (Video): 80 minute presentation on BuzzFeed HTTP routing service abstraction.

• BBC Talks (Slides): different presentations I gave while at the BBC.

• Imager.js (Slides): Talk I gave at the Mozilla offices in Paris (which included speakers from: Google, Apple, Microsoft, Adobe, Opera, W3C and Akamai).

Interviews

• InfoQ: How BuzzFeed Migrated from a Perl Monolith to Go and Python Microservices

Published

I’m a print published and self-published author; I’m also a tech reviewer and am a published author for different popular online organisations (you’ll find lots of technical articles on my own website as well):

Apress

• Pro Vim (Nov 2014)
• Tmux Taster (Nov 2014)
• Quick Clojure: Effective Functional Programming (August 2017)

Packt

• Tech Reviewer Grunt Cookbook (May 2014)
• Tech Reviewer “Troubleshooting Docker” (May 2015)

LeanPub

• Programming in Clojure (Jul 2015)
• Python for Programmers (Jun 2016)

Fastly Blog

Author Overview:
https://www.fastly.com/blog/mark-mcdonnell

• How to configure your Fastly services with Terraform
• Bootstrap a Compute@Edge project with Fastly Fiddle
• Compute@Edge: Go support has arrived!

Fastly dev.to

• Better Fastly API clients with OpenAPI Generator
• How Fastly deploys Gatsby CMS websites to GCS using GitHub Actions

BuzzFeed Tech

• I wrote a three part series on BuzzFeed’s core HTTP routing service (built upon NGINX+) called “Scalable Request Handling: An Odyssey”:
  • Part 1
  • Part 2
  • Part 3

InfoQ

• Interview: How BuzzFeed Migrated from a Perl Monolith to Go and Python Microservices

NET Magazine

• 8 ways to improve your grunt set-up (Nov 2014) (PDF)
• DalekJS vs CasperJS (Nov 2013)

Smashing Magazine

• My author page
• Building Software with Make
• How To Build A CLI Tool With Node.js And PhantomJS
• How To Build A Ruby Gem With Bundler, TDD, Travis CI & Coveralls, Oh My!

NetTuts

• My author page
• Testing Your Ruby Code With Guard, RSpec & Pry (Part 1 - Ruby/Guard/RSpec)
• Testing Your Ruby Code With Guard, RSpec & Pry (Part 2 - RSpec/Pry/Travis-CI)

Popular articles

The following links are to some of my more ‘popular’ articles. My main focus when writing is to take a complicated or confusing topic and attempt to distil it into something that an unfamiliar reader can understand.

• Go Concurrency Patterns (2024)
• Bitwise Operators in Go (2024)
• Continuous Integration and Deployment to multiple environments with Terraform Cloud and GitHub Actions (2024)
• The Power of OpenAPI: Simplifying API Design and documentation (2023)
• Beginner’s guide to creating a Terraform Provider (2022)
• Go Style Guide (2022)
• GitHub Actions (2021)
• Infrequent (and advanced) Vim topics, tips and tricks (2021)
• Git Internals (2020)
• Python Management and Project Dependencies (2019)
• Guide to Concurrency in Python with Asyncio (2019)
• Staying Anonymous (2019)
• HTTP Caching Guide (2019)
• Algorithmic Complexity in Python (2019)
• Data Types and Data Structures (2019)
• Engineer to Manager (2018)
• Interview Techniques (2018)
• Post Mortems (2018)
• Thinking about Interfaces in Go (2018)
• Multigrain Services (2018)
• Authentication with AWS Cognito (2018)
• A guide to effective 1:1 meetings (2018)
• Project Management in Five Minutes (2018)
• Interview Topics (2018)
• Hashing, Encryption and Encoding (2018)
• Computers 101: terminals, kernels and shells (2018)
• Statistics and Graphs: The Basics (2017)
• Observability and Monitoring Best Practices (2017)
• Logging 101 (2017)
• Fastly Varnish (2017)
• Profiling Go (2017)
• Profiling Python (2017)
• Bits Explained (inc. base numbers, ips, cidrs and more) (2016)
• Terminal Debugging Utilities (2016)
• Big O for Beginners (2016)
• Git Merge Strategies (2016)
• HTTP/2 (2015)
• Client Cert Authentication (2015)
• DNS 101 (2015)
• Security basics with GPG, OpenSSH, OpenSSL and Keybase (2015)
• Setting up nginx with Docker (2015)
• Building Software with Make (2015)
• Thread Safe Concurrency (2014)
• GitHub Workflow (2014)
• Understanding recursion in functional JavaScript programming (2014)
• Refactoring Techniques (2013)
• MVCP: Model, View, Controller, Presenter (2013)
• Basic Shell Scripting (2013)
• Object-Oriented Design (OOD) (2013)
• Git Tips (2012)
• JavaScript 101 (2012)

Open-Source

⚠️ NOTE:
I list the following repositories alphabetically (i.e. they are not prioritized)
and they’re probably very stale at this point too.

The reality of life is that I have a family and so, away from work,
I have no time for open-source contributions.

Anything I have worked on has been either done as part of an employer’s
own motivation to contribute to open-source or something I found
5 minutes to spare to share some code I was hacking together.

• BBC Alephant: The Alephant framework is a collection of isolated Ruby gems, which interconnect to offer powerful message passing functionality built up around the “Broker” pattern.

• BBC Imager.js: Responsive images while we wait for srcset to finish cooking

• Bash Headers: CLI tool, written in Bash script, for sorting and filtering HTTP Response Headers

• Carbon: HTTP Response Header Sorting and Filtering written in Go.

• crs: HTTP Response Header Sorting and Filtering written in Rust (supersedes Carbon).

• DOMReady: Cross browser ‘DOM ready’ function

• Go ElastiCache: Thin abstraction over the Memcache client package gomemcache allowing it to support AWS ElastiCache cluster nodes

• Go Fastly CLI: CLI tool, built in Go, for interacting with the Fastly API

• Go Find Root: Locate the root directory of a project using Git via the command line

• Go Flags: Abstraction for command-line flag parsing (with no dependencies except for the Standard Library).

• Go Requester: HTTP service that accepts a collection of “components”, fans-out requests and returns aggregated content

• Go Reverse Proxy: A configuration-driven reverse proxy written in Go (no dependencies except for the standard library).

• Go Web Crawler: A simple web crawler built in Go.

• Grunt Boilerplate: Original Grunt Boilerplate

• Image Slider: HTML5 Canvas Game

• MVCP: MVC + ‘Presenter’ pattern in Ruby

• Terraform Provider Mock: An empty boilerplate repository for creating a terraform provider.

• Sinderella: Ruby gem for transforming data object for specified time frame

• Spurious Clojure AWS SDK Helper: Helper for configuring the AWS SDK to use Spurious

• Squirrel: PhantomJS script to automate Application Cache manifest file generation

• Stark: Node Build Script for serving HTML components

Tools, Languages and Tech

I don’t profess mastery, but I’m adept with most of the below, and I have an aptitude towards learning what I need to get the job done right.

NOTE: With any static list, this will go stale. For example, I’ve not worked with JavaScript for many years (I’m aware of the different client-side platform changes, react, virtual doms etc) and although I was a deep dive expert, that is no longer the case. Take this list with a ‘pinch of salt’.

The slightly more up-to-date list would be:

• Go
• Rust
• Python
• VCL
• Shell Scripting (Bash)
• GitHub Actions (CI/CD pipelines)
• OpenAPI specifications

The longer and more stale version:

• AWS CloudFormation (and the AWS platform in general)
• CSS
• Clojure
• Design Patterns
• Docker
• Functional Programming
• Git
• GitHub Actions (CI/CD)
• Go
• HTML
• JRuby/MRI Ruby
• JavaScript (client-side)
• Jenkins
• Jira
• Make
• Meta Programming (aka Reflection, Macro Systems)
• NGINX
• NSQ
• Node
• PHP
• Python
• Refactoring Techniques
• Regular Expressions
• Rust
• Sass
• Shell Scripting (Bash)
• Terraform
• Tmux
• Trello
• VCL
• Vagrant
• Varnish
• Vim