HubofML #17: Scaling Bert to Serve 1 Billion Requests Per Day, How Instagram CTO Took Its Engineering Team From 0 to 300 and More
As usual, I've compiled a list of my favorite readings from the previous month. I hope you get some fresh thoughts from them. If you like this newsletter, please forward it to others who might be interested.
How to progress or move to the next career ladder is a question almost every engineer had at some point. For an individual contributor, it can even be a lot harder to understand what it means to grow from contributing individually to being an organizational leader. In this article, @xethorn shared what the road to becoming an organizational leader looks like and some valuable questions that will help you think over your path and personal goal.
What comes first when designing a new ML model to solve a business problem? Accuracy or speed? Perhaps, speed before accuracy? In this post, Roblox shared their journey on how they sped up Bert inference by over 30x for text classification. The post contains how they did it while keeping costs manageable.
This is one of the most interesting articles I read last month. An interview with Instagram's CTO Mike Krieger was the basis for this article. He shared lessons learned from growing Instagram's engineering team to 300 people — and what he wishes he could go back and tell himself in 2010. The article contains insights on how to transition from an early to a more mature technical team, how to introduce new tiers of management, and how to build an engine for unrelenting improvement and innovation.
If you're a technical leader, you've probably written (or are about to write) a technical vision for your team or organization. If you’re about to write one, Eventbrite's 3-year technical vision may offer some useful lessons.
Why should you monitor an ML system and what should you monitor? In this post, @jeremyjordan wrote about industry-standard monitoring tools and practices for software systems and how to adapt them to monitor ML systems.
Most organizations often build a data platform to support DS and ML teams for the long term but building a data platform to support both the DS and ML team come with initial challenges. Joseph Bradley, a solution architect at Databricks, found that these often common challenges fall in 3 areas: separation between their data platforms and ML tools, poor communication and collaboration between engineering and DS & ML teams, and past tech choices inhibiting change and growth. The post describes these common mistakes and their solutions distilled into three principles.
“When you join a team, you accept that team’s team's goals, objectives, and priorities. As a principal engineer, you can essentially operate as a one-person team, which makes time management a little more difficult.” Sabrina Leandro, a principal software engineer at Intercoms, outlines how to prioritize goals as a principal engineer in this post
Delta Lake (Delta for short) is an open-source storage layer that brings reliability to your data lake. It does not require you to change your way of working or learn a new API to experience the benefits. In this post, Marijse van den Berg and Maria Zervou wrote about common problems experienced by data scientists and ML engineers and highlights how Delta can alleviate them.
“We have all been there. We see a problem at work and we think we have a solution. Some people may even agree with us and acknowledge the problem with us. So what do we do? we go ahead and try to solve it but suddenly start getting pushback from everywhere and eventually things don’t work out as we had thought they would”. How can we do better? Is there a better way to approach solving a problem? There is something to learn in this post by @kislayverma.
How do you find entrepreneurial engineers who have a business-focused approach to software development? How do you create an environment where they can flourish and make your business take off? How do you nudge great engineers towards focusing more on the business side? In this interview, the VP of Engineering at Opendoor shares how it is done at Opendoor.