My DevOps Story

by
Sneha Ghosh

After completing my undergraduate studies, I worked at Sapient where I was exposed to Agile development technique. As a fresh undergrad, I had never experienced the true utilities of reliable tests and fast releases but it soon became clear to me how important these are to run a production system. I also learned the significance of close collaboration between development, testing and the release team which is absolutely essential to ensure that the incremental changes are rolled out in a smooth fashion. In Maurice Kherlakian’s words, “A phased approach to continuous delivery is not only preferable, it’s infinitely more manageable.”

After working at Sapient for 3 years, I wanted to increase my breadth in the field of Computer Science and I applied for a Master’s Degree at NYU. During my course projects, I tried to employ the same techniques I learned at Sapient to ensure that the incremental changes are tested and it allowed efficient collaboration with my peers. I did an internship at IBM where I worked on automation testing which was used to reliably test and release Db2 Data Warehouse functionalities specifically Snapshot Backup and Restore. Before my work, testing was done using shell scripts. I used a docker testing container which highly simplified the automation setup as it could be easily pulled on a new cluster to kickoff automation. During my internship, I realized that automating a process exposes the deficiencies of the underlying system pretty soon and incentivizes us to fix them. As Bill Gates said, “The first rule of any technology used in a business is that automation applied to an efficient operation will magnify the efficiency. The second is that automation applied to an inefficient operation will magnify the inefficiency.”

During my internship at IBM, I participated in a multi location Hackathon which involved live training provided by IBM software engineers from a variety of domains. We were a team of 5 people working on building an application for crowdfunding and crowdsourcing using blockchain technology. I got a chance to use the Continuous Delivery Pipeline on IBM Cloud to build and deploy the application code. Code commits by all team members were live within seconds which kept our code production ready. As Mike Miller also said, “Simply put, things always had to be in a production-ready state: if you wrote it, you darn well had to be there to get it running!” Given the time crunch and lots of tasks to work on during a hackathon Devops practises saved a lot of time. We were able to test the flow of application easily from time to time. I was really fascinated by such tools which work seamlessly pushing incremental changes automatically after code commits.

I feel that taking the Devops course will help me learn how to use Devops practices to produce a valuable product for the customers. Recent readings about Devops has made me develop a better understanding about it on the whole. I found the following Devops periodic table by xebialabs and am really excited to use some of the tools on a real project.[1]

This course also puts us in an environment where each team is focussing on a different aspect of DevOps, creating an environment like the one we experience in the software industry where everybody works towards a common goal. Such an environment stimulates a faster learning curve through collaboration resulting in a more reliable and sustainable software development.

Sources
  1. Devops Periodic Table
Weekly Updates
  • Week 1 - Learned about the importance of DevOps, challenges of waterfall model and advantages of using Agile/Lean methodology. Also got familiar with the structure of the course and how different teams would collaboratively work on developing the course website using DevOps principles. I am part of the deployment team and our goal is to create the deployment pipeline to automate deployment of course website on cloud or local machine.
    Excited to use tools like Slack, Git, Trello and more later during the course.
  • Week 2 - Learned about git usage and submodules. Published my DevOps Journal.
  • Week 3 - Learned about deploying the application using docker container with a practical exercise. Worked on deployment theory page. Reviewed chapter 1 of Continuous Delivery book.
  • Week 4 - Learned about building modules using make. Reviewed chapters 2 and 3 of Continuous Delivery book. Started studying about puppet installation.
  • Week 5 - Compared SaltStack to other deployment tools and fixed styling issues on the tools page. Created puppet webpage with installation instructions in a client server architechture setup. Reviewed chapter 4 of Continuous Delivery book.
  • Week 6 - Reviewed chapters 5 and 6 of Continuous Delivery book. Studied puppet deployment tutorials and videos to learn writing puppet manifests.
  • Week 7 - Attended presentation by Joe West, CTO of BounceX- software company that provides behavioral automation software and analytics. He talked about 4 important things to know about devops- Business is the thing, right size your toolchain, productivity is the best practice, document and rehearse.
    Worked on the deployment implementation webpage. Provided all steps for setting up puppet master/agent configuration and deploying the website using puppet.
  • Week 8 - Reviewed chapters 7, 8 and 9 of Continuous Delivery book and published the review on the webpage.
  • Week 9 - Reviewed chapter 12 of Continuous Delivery book. Reviewed chapter 1 of Kubernetes Up and Running book. Added example about how to create files from master to agent node for infrastructure configuration using puppet.
  • Week 10 - Reviewed chapter 2 and chapter 3 of Kubernetes Up and Running book. Added section to describe puppet manifest for configuring agents. Started reading about boto3 which can be used for connecting to Amazon EC2 instances using python scripts.
  • Week 11 - Reviewed chapter 7 of Kubernetes Up and Running book. Worked on automating the puppet deployment by writing a script that will create security groups, key pairs and EC2 instances on Amazon and fetch the details of newly created instances.
  • Week 12 - Created scripts for automated installation of puppet packages and configurations on master and agent. Completed the deployment automation scripts for hosting website on Amazon EC2 instaces. Scripts are available in git repository here . Reviewed chapter 11 of Kubernetes Up and Running book.