My overall expectation from this course is to learn all the processes involved in developing and deploying a software the DevOps way. I would love to expand my skill set and gain in-depth practical knowledge of working with containers such as Docker.Also learning how to implement automated testing scripts using Jenkins would be really beneficial since these skills are in huge demand in the industry as of now. Writing scripts to deploy the software on cloud is at the core of DevOps and getting first hand experience doing that would be great. Eventually, I would love if the course prepares us to automate the whole process of deploying to just a single click of a button. All said, I expect the experience of working in a team and collaborating with other teams to be highly beneficial and give us exposure of how things work in industry.
Week 1 and 2 :
The whole class was divided into groups such as coding, workflows, cloud ,deployment, build etc. Due to my excessive interest in coding , i chose to be a part of the coding group. The coding group had 4 different tools to implement out of which one of those was building a spell checker tool. Me and Kanishk decided to work and collaborate on this project and one of the major reasons i chose this tool was because of my interest in Natural Language Understanding(NLU). I believe i could put to use a lot of domain knowledge on the subject along with my excellent coding skills to build the tool in python.
Also we learnt how to use trello for managing the workflow of the projects and updated it by adding a trello card for HTML spell checker in the wish list column.
Week 3:
Attended a wonderful session on Docker by Prashant Patel which was a very practical and hands-on session. I learnt the practical usage of docker and bunch of commands on how to start and and manage docker conatainers.Learnt how to work with nginx. Some of the commands we executed in the terminal were as follows :
docker start, docker ps,docker images , docker pull nginx etc.
We also started the development work for spell checker tool by first performing a requirements analysis of the tool for which we also met with professor to clarify the tool requirements and the features it should possess. After that we wrote down the code containing the basic features of the spell checker and made our first commit for the tool.
Week 4 :
We added more features to the spell checker tool such as the functionality for the user to be able to add words to the dictionary that dont exist beforehand. Also, we met with professor to discuss the project status where he gave us a bunch of suggestions as well on how we could improve the tool.
Week 5 :
Got a chance to attend a very fruitful and informative session by Tom Limoncelli who's a Site reliability Engineer at StackOverflow. He gave a very detailed insight into how DevOps really works in industry. There were 3 main main takeaways from his presentation about the DevOps principles which were as follows:
-> "The Three ways of DevOps" : The Process, Communication and Trying New Things.He believes that an organization can increase their throughput if they focus on improving the above three criteria.
-> The Small Batches Principle : Doing work in small batches or increments is always better than developing the whole software and deploying only after completion. It ensures that feedback from all the stakeholders in the process is obtained at an early stage rather than at the end.
-> Minimum Viable Product: Delivering some value sooner is better than all value later.
Week 6 :
Attended a talk by Tom Germano from Google who gave a deep insight about how Web Search works also how results are served to search queries. One of his primary responsibilities is to handle joins on databases which becomes a complex task when performing info update operations on millions of servers concurrently.
Week 7 :
Spring Break
Week 8 :
Attended a very informational session by Joe West, Former CTO of BounceX who gave a deep insight about how devops is being adopted in the industry and also shared his experiences about Akamai. Some of the main take away concepts from his presentation were as follows:
Productivity is the most important thing: He gave an example of a lazy systems engineer who is better than a more active one because he will always find ways to automate things.
Focus on the business value: Don't use DevOps tools just because the technology looks cool, assess the customer requirements and whether it adds value to business and will be profitable in the future to make decision on whether it makes sense to use those services.
Using the right toolkit : Optimizee the workflow to understand the minimum amount of resources needed to fulfill the requirement. Keep it simple.
Week 9 :
This week we added functionality in the spell checker tool to use regular expressions and skip any words containing characters other than alphabets or hyphen.
Week 10 :
This week we added functionality to skip all the content inside code tags in the spell checker tool. Also started reading chapter 1 of Practical DevOps by Joakim Verona.
Week 11 :
Attended the very informative lecture about the importance and application of security in the DevOps pipeline. Also wrote book review for Chapter 1 - Introduction to DevOps and Continuous Delivery. Worked on refactoring the code of spell checker tool and removing redundant parts in the code.
Week 12 :
Deployed the spell checker tool on the website and fixed issues in the shell script which executes the tool .Updated the journal and
read chapter 2 of Practical Devops.
Week 13 :
Took the decision to do partial rollout of the tool instead of rolling it out for everyone at once due to arising bugs. Basically focused on 4 tasks during the week :
1. Moved the custom dictionary from utils repo to DevOps repo.
2. Added the class roster to custom dictionary so that the tool recognizes names of class students.
3. Added a wrapper shell script to automate : executes the spell checker program , commits the custom dictionary to git, and subsequently returns the exit code to the calling shell script.
4. Added partial rollout feature by checking $USER in the shell script and rolled out the tool to only spell checker team and Professor Callahan.
Week 14 :
This week I presented the book review for Practical DevOps in class and gave all the students insight into why they should read this book.
During the week I made the following bug fix in the spell checker tool:
Problem : The tool wasn't recognizing some valid english words and was prompting the user for too many words which made the whole task laborious and the tool , virtually unfit for production.
Diagnosis : The cause was diagnosed as the dictionary source not being very elaborate and just containing 10k words.
Fix : I added an extensive dictionary source containing more than 300k words , which covered all the root forms of all possible words. Still, many inflections(tenses , plurals) of words were not being covered by this source and the presence of verified sources of dictionary covering everything as well as being free for download is rare on the internet, hence difficult to procure. So, I came up with a solution to add a backup dictionary source which is verified, trusted and popular. I did some research and came across Oxford Dictionary REST API which returns a boolean value on sending a word in the request URL.That instantly did the trick and I successfully integrated the API into the spell checker tool. Now the tool accuracy is almost near perfect. The decision to make the Oxford API as a backup source and not the sole source was taken because it has a limit of 60 requests/min in the free plan. Hence the cost factor was also taken into account in designing the Software Architecture.
Week 15 :
Read remaining chapters from the Practical DevOps book and wrote book reviews for them subsequently.