ML for Game Development, the Trimodal Nature of Software Engineer Salaries, Turning Around Underperforming Software Teams and More
Featured posts from GoogleAI, Uber, LinkedIn, Facebook, Dropbox, Shopify, GetYourGuide, and more
As usual, I've compiled a list of my favorite discoveries from the previous month. I hope this edition inspires you to think creatively. If you missed the last one, you could catch up by clicking here.
If you like this newsletter, please forward it to others who might be interested.
For Machine Learning Engineers
Recently, LinkedIn open-sourced LinkedIn Fairness Toolkit (LiFT), a Scala/Spark library, allows you to measure fairness in large-scale machine learning workflows. LiFT enables you to measure fairness metrics on training data and for model performance. You can check it out here.
Big Data and Data Science is at the heart of Uber, powering everything the company does, including better pricing and matching, fraud detection, shorter ETAs, and experimentation. However, at the Uber scale, data duplication, data discovery, data ownership, inconsistency in logging, and other issues did not go away. Data problems are just as common in most tech companies as they are in Uber, which is why I found this post interesting and insightful.
Having a transparent and straightforward workflow for productionizing ML models is crucial to reducing human errors. That’s why I think you might find this post interesting on how GetYourGuide is tackling this problem. Theodore Meynard a senior data scientist at GetYourGuide, shares the foundational principles for building the GetYourGuide’s machine learning platform leveraging open-source software to bring transparency and speed to ML models’ productionalization at GetYourGuide.
A feature store is a repository where you can store, update, retrieve, and share machine learning (ML) features. The concept of feature store is gaining traction in the field of machine learning system operations. But how do you design a feature store, and when do you need a feature store?. In this post, @neal_lathia, an ML director at Monzo, shares some of the thinking that motivated his team towards building a feature store and how he designed it.
Game developers often stress-test through thousands of play-testing sessions from test users before launch. With this, they obtain feedback and improve the gaming experience. But what if you could train ML models to serve as test testers? This post from the GoogleAI blog introduces an approach that leverages machine learning (ML) to adjust game balance by training models to serve as play-tester
For Engineering Leads
It's natural to have questions and self-doubts when starting a new job or new role in a new company. What if there are steps you can take to increase your chances of success in a new role? In this post, @korndaniel1 outlines four things he did to become successful as Engineering Team Lead after joining Lemonade.
“As you advance into engineering management, your soft skills as a leader will make or break your career. Many engineering leaders give up and turn back, while others succeed. Many of us believe that successful people were born with the soft skills required for leadership”. This story by @felhobacsi, VP of Engineering at Bitrise will tell you otherwise. Every tech leader should read this.
How do you improve software engineering team performance? Every engineering leader wants to know the answer to this question. It is especially important when taking over an underperforming team. How do you make it right? How can engineering productivity be increased? This post describes the strategy and framework used and developed by LogmeIn's VP of Engineering, John Ford.
A Checklist For First-Time Engineering Managers
“The most noticeable difference in moving into management is that you get a lot less feedback. When you're an engineer, you get feedback on your code, design documents, and how your projects progress. You do not have any of these as a manager. There are also frequently no clear and specific expectations of what you should be doing in this role.” @GergelyOrosz created this helpful checklist for first-time engineering managers.
For Software Engineers
The Trimodal Nature of Software Engineering Salaries in the Netherlands and Europe
Gergely Orosz's pragmatic engineer blog is one of the few I read. @GergelyOrosz explains in this post how engineering compensation has shifted from a market average to three distinct groups that "spike" and have little overlap. According to him, "there is no longer an "average" salary for software engineers in Europe / the Netherlands: only an average salary in one of three distinct categories."
“The entire Facebook ecosystem is powered by thousands of distributed systems and microservices, many of which would benefit from running the workload asynchronously, particularly at peak times of online traffic.” Recently, Facebook shares the design of their Ordered Queue System, with the hope that the broader engineering community can build upon the ideas and designs.
Most engineers have worked with slow CI at some point in their careers. I've been there, and it kills agility and productivity. Even if your team does not use Buildkite like Shopify, there are lessons in this post for everyone on how Shopify’s Test Infrastructure team was able to reduce the p95 of Shopify's core monolith CI from 45 minutes to 18 minutes.
A Guide to Creating API Products
It takes more than just returning responses to create a good Application Programming Interface (API). As a developer who has integrated with hundreds of APIs, I've noticed a pattern between successful and unsuccessful API products. It's all about solving problems with great affordance. One might wonder what the key to creating good API products is. This post contains some helpful hints for developing API products.
Dropbox’s engineers explain their journey from Python Monolith to developing and using Atlas. Atlas provides the majority of benefits of a Service Oriented Architecture while minimizing the operational cost that typically comes with owning a service.
Thanks for reading! If you like this newsletter and want to support it, please share it with others.