With the rise of DevOps, low-cost cloud computing, and container technologies, the way Java developers approach development today has changed dramatically. This practical guide helps you take advantage of microservices, serverless, and cloud native technologies using the latest DevOps techniques to simplify your build process and create hyperproductive teams.
Stephen Chin, Melissa McKay, Ixchel Ruiz, and Baruch Sadogursky help you evaluate an array of options. The list includes source control with Git, build declaration with Maven and Gradle, CI/CD with CircleCI, package management with Artifactory, containerization with Docker and Kubernetes, and much more. Whether you’re building applications with Jakarta EE, Spring Boot, Dropwizard, MicroProfile, Micronaut, or Quarkus, this comprehensive guide has you covered.
Explore software lifecycle best practices
Use DevSecOps methodologies to facilitate software development and delivery
Understand the business value of DevSecOps best practices
Manage and secure software dependencies
Develop and deploy applications using containers and cloud native technologies
Manage and administrate source control repositories and development processes
Use automation to set up and administer build pipelines
Identify common deployment patterns and antipatterns
Maintain and monitor software after deployment
About the Author
Stephen Chin is Head of Developer Relations at JFrog and author of The Definitive Guide to Modern Client Development, Raspberry Pi with Java, and Pro JavaFX Platform. He has keynoted numerous Java conferences around the world including Devoxx, JNation, JavaOne, Joker, and Open Source India. Stephen is an avid motorcyclist who has done evangelism tours in Europe, Japan, and Brazil, interviewing hackers in their natural habitat. When he is not traveling, he enjoys teaching kids how to do embedded and robot programming together with his teenage daughter. You can follow his hacking adventures at: http://steveonjava.com/.
Melissa McKay is currently a Developer Advocate with the JFrog Developer Relations team. She has been active in the software industry 20 years and her background and experience spans a slew of technologies and tools used in the development and operation of enterprise products and services. Melissa is a mom, software developer, Java geek, huge promoter of Java UNconferences, and is always on the lookout for ways to grow, learn, and improve development processes. She is active in the developer community, has spoken at CodeOne, Java Dev Day Mexico and assists with organizing the JCrete and JAlba Unconferences as well as Devoxx4Kids events.
Ixchel Ruiz has developed software applications and tools since 2000. Her research interests include Java, dynamic languages, client-side technologies, and testing. She is a Java Champion, Groundbreaker Ambassador, Hackergarten enthusiast, open source advocate, JUG leader, public speaker, and mentor.
Baruch Sadogursky (a.k.a JBaruch) is the Chief Sticker Officer @JFrog (also, Head of DevOps Advocacy) at JFrog. His passion is speaking about technology. Well, speaking in general, but doing it about technology makes him look smart, and 19 years of hi-tech experience sure helps. When he’s not on stage (or on a plane to get there), he learns about technology, people and how they work, or more precisely, don’t work together.
He is a co-author of the Liquid Software book, a CNCF ambassador and a passionate conference speaker on DevOps, DevSecOps, digital transformation, containers and cloud-native, artifact management and other topics, and is a regular at the industry’s most prestigious events including DockerCon, Devoxx, DevOps Days, OSCON, Qcon, JavaOne and many others. You can see some of his talks at jfrog.com/shownotes
See: Beginning Git and GitHub: A Comprehensive Guide to Version Control, Project Management, and Teamwork for the New Developer 1st ed. Edition, Kindle Edition
“A DevOps toolchain is a set or combination of tools that aid in the delivery, development, and management of software applications throughout the systems development life cycle, as coordinated by an organization that uses DevOps practices.
“In software, a toolchain is the set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs. In general, the tools forming a toolchain are executed consecutively so the output or resulting environment state of each tool becomes the input or starting environment for the next one, but the term is also used when referring to a set of related tools that are not necessarily executed consecutively.[3][4][5]
As DevOps is a set of practices that emphasizes the collaboration and communication of both software developers and other information technology (IT) professionals, while automating the process of software delivery and infrastructure changes, its implementation can include the definition of the series of tools used at various stages of the lifecycle; because DevOps is a cultural shift and collaboration between development and operations, there is no one product that can be considered a single DevOps tool. Instead a collection of tools, potentially from a variety of vendors, are used in one or more stages of the lifecycle.[6][7]” (WP)
Plan is composed of two things: “define” and “plan”.[8] This activity refers to the business value and application requirements. Specifically “Plan” activities include:
Tools and vendors in this category often overlap with other categories. Because DevOps is about breaking down silos, this is reflective in the activities and product solutions.[clarification needed]
Verify
Verify is directly associated with ensuring the quality of the software release; activities designed to ensure code quality is maintained and the highest quality is deployed to production.[8] The main activities in this are:
Packaging refers to the activities involved once the release is ready for deployment, often also referred to as staging or Preproduction / “preprod”.[8] This often includes tasks and activities such as:
Approval/preapprovals
Package configuration
Triggered releases
Release staging and holding
Release
Release related activities include schedule, orchestration, provisioning and deploying software into production and targeted environment.[9] The specific Release activities include:
Configure activities fall under the operation side of DevOps. Once software is deployed, there may be additional IT infrastructure provisioning and configuration activities required.[8] Specific activities including:
Infrastructure storage, database and network provisioning and configuring
Monitoring is an important link in a DevOps toolchain. It allows IT organization to identify specific issues of specific releases and to understand the impact on end-users.[8] A summary of Monitor related activities are:
Information from monitoring activities often impacts Plan activities required for changes and for new release cycles.
Version Control
Version Control is an important link in a DevOps toolchain and a component of software configuration management. Version Control is the management of changes to documents, computer programs, large web sites, and other collections of information.[8] A summary of Version Control related activities are:
Non-linear development
Distributed development
Compatibility with existent systems and protocols
Toolkit-based design
Information from Version Control often supports Release activities required for changes and for new release cycles.
^ Garner Market Trends: DevOps – Not a Market, but Tool-Centric Philosophy That supports a Continuous Delivery Value Chain (Report). Gartner. 18 February 2015.
^ abcdefg Avoid Failure by Developing a Toolchain that Enables DevOps (Report). Gartner. 16 March 2016.
^ Best Practices in Change, Configuration and Release Management (Report). Gartner. 14 July 2010.
^ Roger S. Pressman (2009). Software Engineering: A Practitioner’s Approach (7th International ed.). New York: McGraw-Hill.
GitHub offers its basic services free of charge. Its more advanced professional and enterprise services are commercial.[5] Free GitHub accounts are commonly used to host open-source projects.[6] As of January 2019, GitHub offers unlimited private repositories to all plans, including free accounts, but allowed only up to three collaborators per repository for free.[7] Starting from April 15, 2020, the free plan allows unlimited collaborators, but restricts private repositories to 2,000 minutes of GitHub Actions[8] per month.[9] As of January 2020, GitHub reports having over 40 million users[10] and more than 190 million repositories[11] (including at least 28 million public repositories),[12] making it the largest host of source code in the world.[13]
History
GitHub at AWS Summit
The GitHub service was developed by Chris Wanstrath, P. J. Hyett, Tom Preston-Werner and Scott Chacon using Ruby on Rails, and started in February 2008. The company, GitHub, Inc., has existed since 2007 and is located in San Francisco.[14]The shading of the map illustrates the number of users as a proportion of each country’s Internet population. The circular charts surrounding the two hemispheres depict the total number of GitHub users (left) and commits (right) per country.
On February 24, 2009, GitHub announced that within the first year of being online, GitHub had accumulated over 46,000 public repositories, 17,000 of which were formed in the previous month. At that time, about 6,200 repositories had been forked at least once and 4,600 had been merged.
That same year, the site was harnessed by over 100,000 users, according to Github, and had grown to host 90,000 unique public repositories, 12,000 having been forked at least once, for a total of 135,000 repositories.[15]
In 2010, GitHub was hosting 1 million repositories.[16] A year later, this number doubled.[17]ReadWriteWeb reported that GitHub had surpassed SourceForge and Google Code in total number of commits for the period of January to May 2011.[18] On January 16, 2013, GitHub passed the 3 million users mark and was then hosting more than 5 million repositories.[19] By the end of the year, the number of repositories were twice as much, reaching 10 million repositories.[20]
In 2012, GitHub raised $100 million in funding from Andreessen Horowitz with $750 million valuation.[21]Peter Levine, general partner at Andreessen Horowitz, stated that GitHub had been growing revenue at 300% annually since 2008 “profitably nearly the entire way”.[22] On July 29, 2015, GitHub stated it had raised $250 million in funding in a round led by Sequoia Capital. Other investors of that round included Andreessen Horowitz, Thrive Capital, and IVP (Institutional Venture Partners).[23] The round valued the company at approximately $2 billion.[24]
In 2015, GitHub opened an office in Japan that is its first office outside of the U.S.[25] In 2016, GitHub was ranked No. 14 on the Forbes Cloud 100 list.[26] It has not been featured on the 2018, 2019 and 2020 lists.[27]
On February 28, 2018, GitHub fell victim to the third largest distributed denial-of-service (DDoS) attack in history, with incoming traffic reaching a peak of about 1.35 terabits per second.[28]
On June 19, 2018, GitHub expanded its GitHub Education by offering free education bundles to all schools.[29][30]
On June 4, 2018, Microsoft announced its intent to acquire GitHub for US$7.5 billion. The deal closed on October 26, 2018.[33] GitHub continued to operate independently as a community, platform and business.[34] Under Microsoft, the service was led by Xamarin‘s Nat Friedman, reporting to Scott Guthrie, executive vice president of Microsoft Cloud and AI. GitHub’s CEO, Chris Wanstrath, was retained as a “technical fellow”, also reporting to Guthrie.
There have been concerns from developers Kyle Simpson, JavaScript trainer and author, and Rafael Laguna, CEO at Open-Xchange over Microsoft’s purchase, citing uneasiness over Microsoft’s handling of previous acquisitions, such as Nokia’s mobile business or Skype.[35][36]
This acquisition was in line with Microsoft’s business strategy under CEO Satya Nadella, which has seen a larger focus on the cloud computing services, alongside development of and contributions to open-source software.[37][4][32]Harvard Business Review argued that Microsoft was intending to acquire GitHub to get access to its user base, so it can be used as a loss leader to encourage use of its other development products and services.[38]
Concerns over the sale bolstered interest in competitors: Bitbucket (owned by Atlassian), GitLab (a commercial open source product that also runs a hosted service version) and SourceForge (owned by BIZX, LLC) reported that they had seen spikes in new users intending to migrate projects from GitHub to their respective services.[39][40][41][42]
In September 2019, GitHub acquired Semmle, a code analysis tool.[43] In February 2020, GitHub launched in India under the name GitHub India Private Limited.[44] In March 2020, GitHub announced that they were acquiring npm, a JavaScript packaging vendor, for an undisclosed sum of money.[45] The deal was closed on 15 April 2020.[46]
In early July 2020, the GitHub Archive Program was established, to archive its open source code in perpetuity.[47]
Services
GitHub.com
Development of the GitHub.com platform began on October 19, 2007.[60][61][62] The site was launched in April 2008 by Tom Preston-Werner, Chris Wanstrath, P. J. Hyett and Scott Chacon after it had been made available for a few months prior as a beta release.[63]
Projects on GitHub.com can be accessed and managed using the standard Git command-line interface; all standard Git commands work with it. GitHub.com also allows users to browse public repositories on the site. Multiple desktop clients and Git plugins are also available. The site provides social networking-like functions such as feeds, followers, wikis (using wiki software called Gollum) and a social network graph to display how developers work on their versions (“forks“) of a repository and what fork (and branch within that fork) is newest.
Anyone can browse and download public repositories but only registered users can contribute content to repositories. With a registered user account, users are able to have discussions, manage repositories, submit contributions to others’ repositories, and review changes to code. GitHub.com began offering unlimited private repositories at no cost in January 2019 (limited to three contributors per project). Previously, only public repositories were free.[64][65][66] On April 14, 2020, GitHub made “all of the core GitHub features” free for everyone, including “private repositories with unlimited collaborators”.[67]
The fundamental software that underpins GitHub is Git itself, written by Linus Torvalds, creator of Linux. The additional software that provides the GitHub user interface was written using Ruby on Rails and Erlang by GitHub, Inc. developers Wanstrath,[68] Hyett, and Preston-Werner.
Scope
The main purpose of GitHub.com is to facilitate the version control and issue tracking aspects of software development. Labels, milestones, responsibility assignment, and a search engine are available for issue tracking. For version control, Git (and by extension GitHub.com) allows pull requests to propose changes to the source code. Users with the ability to review the proposed changes can see a diff of the requested changes and approve them. In Git terminology, this action is called “committing” and one instance of it is a “commit”. A history of all commits are kept and can be viewed at a later time.
In addition, GitHub supports the following formats and features:
Documentation, including automatically rendered README files in a variety of Markdown-like file formats (see README § On GitHub)
GitHub Actions, which allows building continuous integration and continuous deployment pipelines for testing, releasing and deploying software without the use of third-party websites/platforms
Graphs: pulse, contributors, commits, code frequency, punch card, network, members
3D render files that can be previewed using a new integrated STL file viewer that displays the files on a “3D canvas”.[72] The viewer is powered by WebGL and Three.js.
Photoshop’s native PSD format can be previewed and compared to previous versions of the same file.
GitHub’s Terms of Service do not require public software projects hosted on GitHub to meet the Open Source Definition. The terms of service state, “By setting your repositories to be viewed publicly, you agree to allow others to view and fork your repositories.”[73]
GitHub Enterprise
GitHub Enterprise is a self-managed version of GitHub.com with similar functionality. It can be run on an organization’s own hardware or on a cloud provider, and it has been available since November 2011.[74] In November 2020, source code for GitHub Enterprise Server was leaked online in apparent protest against DMCA takedown of YouTube-dl. According to GitHub, the source code came from GitHub accidentally sharing the code with Enterprise customers themselves, not from an attack on GitHub servers.[75][76]
GitHub Pages
GitHub Pages is a staticweb hosting service offered by GitHub since 2008 to GitHub users for hosting user blogs, project documentation,[77][78] or even whole books created as a page.[79]
All GitHub Pages content is stored in a Git repository, either as files served to visitors verbatim or in Markdown format. GitHub is seamlessly integrated with Jekyll static web site and blog generator and GitHub continuous integration pipelines. Each time the content source is updated, Jekyll regenerates the website and automatically serves it via GitHub Pages infrastructure.[80]
As with the rest of GitHub, it includes both free and paid tiers of service, instead of being supported by web advertising. Web sites generated through this service are hosted either as subdomains of the github.io domain, or as custom domains bought through a third-party domain name registrar.[81] When custom domain is set on a GitHub Pages repo a Let’s Encrypt certificate for it is generated automatically. Once the certificate has been generated Enforce HTTPS can be set for the repository’s website to transparently redirect all HTTP requests to HTTPS.[82][83]
Gist
GitHub also operates other services: a pastebin-style site called Gist[63] that is for hosting code snippets (GitHub proper is for hosting larger projects).
Tom Preston-Werner presented the then-new Gist feature at a punk rock Ruby conference in 2008.[84] Gist builds on the traditional simple concept of a pastebin by adding version control for code snippets, easy forking, and TLS encryption for private pastes. Because each “gist” has its own Git repository, multiple code snippets can be contained in a single paste and they can be pushed and pulled using Git. Further, forked code can be pushed back to the original author in the form of a patch, so gists (pastes) can become more like mini-projects.[citation needed]
Before February 18, 2018, unregistered users were able to upload text to the site. Since then, uploading gists has been deactivated for unregistered users with the aim to mitigate spamming.[85]
In 2016 GitHub announced the launch of the GitHub Campus Experts program[87] to train and encourage students to grow technology communities at their universities. The Campus Experts program is open to university students of 18 years and older across the world.[88] GitHub Campus Experts are one of the primary ways that GitHub funds student oriented events and communities, Campus Experts are given access to training, funding, and additional resources to run events and grow their communities. To become a Campus Expert applicants must complete an online training course consisting of multiple modules designed to grow community leadership skills.
GitHub Marketplace service
GitHub also provides some software as a service integrations for adding extra features to projects. Those services include:
Waffle.io: Project management for software teams. Automatically see pull requests, automated builds, reviews, and deployments across all of your repositories in GitHub.
Rollbar: Integrate with GitHub to provide real time debugging tools and full-stack exception reporting. It is compatible with all well used code languages, such as JavaScript, Python, .NET, Ruby, PHP, Node.js, Android, iOS, Go, Java, and C#.
Travis CI: To provide confidence for your apps while doing test and ship. Also gives full control over the build environment, to adapt it to the code. Supported languages: Go, Java, JavaScript, Objective-C, Python, PHP, Ruby, and Swift.
GitLocalize: Developed for teams that are translating their content from one point to another. GitLocalize automatically syncs with your repository so you can keep your workflow on GitHub. It also keeps you updated on what needs to be translated.
GitHub Sponsors
GitHub Sponsors allows users to make monthly money donations to projects hosted on GitHub.[89] The public beta was announced on May 23, 2019 and currently the project accepts wait list registrations. The Verge said that GitHub Sponsors “works exactly like Patreon” because “developers can offer various funding tiers that come with different perks, and they’ll receive recurring payments from supporters who want to access them and encourage their work” except with “zero fees to use the program”. Furthermore, GitHub offer incentives for early adopters during the first year: it pledges to cover payment processing costs, and match sponsorship payments up to $5,000 per developer. Furthermore, users still can use other similar services like Patreon and Open Collective and link to their own websites.[90][91]
GitHub Archive Program
In July 2020, GitHub stored a February archive of the site[47] in an abandoned mountain mine in Svalbard, Norway, part of the Arctic World Archive and not far from the Svalbard Global Seed Vault. The archive contained the code of all active public repositories, as well as that of dormant, but significant public repositories. The 21TB of data was stored on piqlFilm archival film reels as QR codes, and is expected to last 500–1,000 years.[92][93][94][95]
The GitHub Archive Program is also working with partners on Project Silica, in an attempt to store all public repositories for 10,000 years. It aims to write archives into the molecular structure of quartz glass platters, using a high-precision laser that pulses a quadrillion (1,000,000,000,000,000) times per second.[95]
Developed projects
Atom, a free and open-source text and source code editor
Electron, an open-source framework to use JavaScript-based websites as desktop applications.
Prominent users
Some prominent open source organizations and projects use GitHub as a primary place for collaboration, including:
