Yahoo Open Sources Screwdriver to Enable Continuous Delivery

21st January 2017 | By Mouseworld Now Correspondent |

yahoo-new-logoNew Delhi, January 21, 2017: Yahoo has announced the open sourcing of Screwdriver.cd, a new streamlined build system designed to enable continuous delivery (CD) to production at scale for dynamic infrastructure, improving product quality and engineering agility.

Continuous Delivery enables software development teams to move faster and adapt to users’ needs quicker by reducing the inherent friction associated with releasing software changes. Yahoo’s engineering has modernized as it has embraced Continuous Delivery as a strategy for improving product quality and engineering agility. All Yahoo active products deliver from commit to production with full automation and this has greatly improved Yahoo’s ability to deliver products.

Part of what enabled Yahoo to make Continuous Delivery at scale a reality was its improved build and release tooling. Now, Yahoo is open sourcing an adaptation of its code as Screwdriver.cd. Some of the key design features of Screwdriver have helped Yahoo achieve Continuous Delivery at scale. At a high level these are making deployment pipelines easy; optimizing for trunk development; and making rolling back easy.

Deployment pipelines that continuously test, integrate, and deploy code to production greatly reduce the risk of errors and reduce the time to get feedback to developers. The challenge for many groups had been that pipelines were cumbersome to setup and maintain. We designed a solution that made pipelines easy to configure and completely self-service for any developer. By managing the pipeline configuration in the code repository Screwdriver allows developers to configure pipelines in a manner familiar to them, and as a bonus, to easily code review pipeline changes too.

Internally, we encourage workflows where the trunk is always shippable. Our teams use a modified GitHub flow for their workflows. Pull Requests (PRs) are the entry point for running tests and ensuring code that entered the repository has been sufficiently tested. Insisting on formal PRs also improves the quality of our code reviews.

To ensure trunks are shippable, we enable functional testing of code in the PRs. Internally, this is a configuration baked into pipelines that dynamically allocates compute resources, deploys the code, and runs tests. These tests include web testing using tools like Selenium. These dynamically-allocated resources are also available for a period after the PR build, allowing engineers to interact with the system and review visual aspects of their changes.

To allow for easy code rollbacks, Yahoo allow phases of the pipeline to be re-run at a previously-saved state. We leverage features in its PaaS to handle the deployment, but we store and pass metadata to enable us to re-run from a specific git SHA with the same deployment data. This allows Yahoo to roll back to a previous state in production. This design makes rolling back as easy as selecting a version from a dropdown menu and clicking “deploy.” Anyone with write access to the project can make this change. This helped Yahoo move teams to a DevOps model where developers were responsible for the production state.

The successful growth of Screwdriver over the past 5 years at Yahoo has today led to Screwdriver being synonymous with Continuous Delivery within the company. Screwdriver handles over 25,000+ builds per day and 12,000+ daily git commits as a single shared entrypoint for Yahoo. It supports multiple languages and handles both virtual machine and container-based builds and deployment.

Tags: , , , , , , ,

Leave your comment

IMPORTANT! To be able to proceed, you need to solve the following simple math

What is 9 + 8 ?
Please leave these two fields as-is:

Mouseworldnow Videos
  • r chandrashekhar president nasscom
  • Anant Maheshwari, President, Microsoft India
  • Suresh_Vaswani-220 by 220

Channel News

  • Rashi Peripherals Conducts Multi-City Enterprise Partner Meet more...
  • Second Edition of Annual 4.5G & 5G Innovation Summit concludes on a successful note more...
  • Wydr launches India Wholesale E-Fair more...
  • Snapdeal clocks record 3x growth in Kids Category more...
  • Paytm ensures 100% security for users’ identity in Money transfers more...
Subscribe via email

Enter your email address:

Follow us on Facebook
QUESTION HOUR
What does the mouse ask?

Will the spurt in online video advertisement steal the twinkle from the TV ad platform?

View Results

Loading ... Loading ...
Newsletter Registration