With the rise of DevOps, low-cost cloud computing, and container technologies, the way Java developers approach development today has changed dramatically. This practical guide helps you take advantage of microservices, serverless, and cloud native technologies using the latest DevOps techniques to simplify your build process and create hyperproductive teams.
Stephen Chin, Melissa McKay, Ixchel Ruiz, and Baruch Sadogursky help you evaluate an array of options. The list includes source control with Git, build declaration with Maven and Gradle, CI/CD with CircleCI, package management with Artifactory, containerization with Docker and Kubernetes, and much more. Whether you’re building applications with Jakarta EE, Spring Boot, Dropwizard, MicroProfile, Micronaut, or Quarkus, this comprehensive guide has you covered.
Explore software lifecycle best practices
Use DevSecOps methodologies to facilitate software development and delivery
Understand the business value of DevSecOps best practices
Manage and secure software dependencies
Develop and deploy applications using containers and cloud native technologies
Manage and administrate source control repositories and development processes
Use automation to set up and administer build pipelines
Identify common deployment patterns and antipatterns
Maintain and monitor software after deployment
About the Author
Stephen Chin is Head of Developer Relations at JFrog and author of The Definitive Guide to Modern Client Development, Raspberry Pi with Java, and Pro JavaFX Platform. He has keynoted numerous Java conferences around the world including Devoxx, JNation, JavaOne, Joker, and Open Source India. Stephen is an avid motorcyclist who has done evangelism tours in Europe, Japan, and Brazil, interviewing hackers in their natural habitat. When he is not traveling, he enjoys teaching kids how to do embedded and robot programming together with his teenage daughter. You can follow his hacking adventures at: http://steveonjava.com/.
Melissa McKay is currently a Developer Advocate with the JFrog Developer Relations team. She has been active in the software industry 20 years and her background and experience spans a slew of technologies and tools used in the development and operation of enterprise products and services. Melissa is a mom, software developer, Java geek, huge promoter of Java UNconferences, and is always on the lookout for ways to grow, learn, and improve development processes. She is active in the developer community, has spoken at CodeOne, Java Dev Day Mexico and assists with organizing the JCrete and JAlba Unconferences as well as Devoxx4Kids events.
Ixchel Ruiz has developed software applications and tools since 2000. Her research interests include Java, dynamic languages, client-side technologies, and testing. She is a Java Champion, Groundbreaker Ambassador, Hackergarten enthusiast, open source advocate, JUG leader, public speaker, and mentor.
Baruch Sadogursky (a.k.a JBaruch) is the Chief Sticker Officer @JFrog (also, Head of DevOps Advocacy) at JFrog. His passion is speaking about technology. Well, speaking in general, but doing it about technology makes him look smart, and 19 years of hi-tech experience sure helps. When he’s not on stage (or on a plane to get there), he learns about technology, people and how they work, or more precisely, don’t work together.
He is a co-author of the Liquid Software book, a CNCF ambassador and a passionate conference speaker on DevOps, DevSecOps, digital transformation, containers and cloud-native, artifact management and other topics, and is a regular at the industry’s most prestigious events including DockerCon, Devoxx, DevOps Days, OSCON, Qcon, JavaOne and many others. You can see some of his talks at jfrog.com/shownotes
See: Beginning Git and GitHub: A Comprehensive Guide to Version Control, Project Management, and Teamwork for the New Developer 1st ed. Edition, Kindle Edition
“A DevOps toolchain is a set or combination of tools that aid in the delivery, development, and management of software applications throughout the systems development life cycle, as coordinated by an organization that uses DevOps practices.
“In software, a toolchain is the set of programming tools that is used to perform a complex software development task or to create a software product, which is typically another computer program or a set of related programs. In general, the tools forming a toolchain are executed consecutively so the output or resulting environment state of each tool becomes the input or starting environment for the next one, but the term is also used when referring to a set of related tools that are not necessarily executed consecutively.[3][4][5]
As DevOps is a set of practices that emphasizes the collaboration and communication of both software developers and other information technology (IT) professionals, while automating the process of software delivery and infrastructure changes, its implementation can include the definition of the series of tools used at various stages of the lifecycle; because DevOps is a cultural shift and collaboration between development and operations, there is no one product that can be considered a single DevOps tool. Instead a collection of tools, potentially from a variety of vendors, are used in one or more stages of the lifecycle.[6][7]” (WP)
Plan is composed of two things: “define” and “plan”.[8] This activity refers to the business value and application requirements. Specifically “Plan” activities include:
Tools and vendors in this category often overlap with other categories. Because DevOps is about breaking down silos, this is reflective in the activities and product solutions.[clarification needed]
Verify
Verify is directly associated with ensuring the quality of the software release; activities designed to ensure code quality is maintained and the highest quality is deployed to production.[8] The main activities in this are:
Packaging refers to the activities involved once the release is ready for deployment, often also referred to as staging or Preproduction / “preprod”.[8] This often includes tasks and activities such as:
Approval/preapprovals
Package configuration
Triggered releases
Release staging and holding
Release
Release related activities include schedule, orchestration, provisioning and deploying software into production and targeted environment.[9] The specific Release activities include:
Configure activities fall under the operation side of DevOps. Once software is deployed, there may be additional IT infrastructure provisioning and configuration activities required.[8] Specific activities including:
Infrastructure storage, database and network provisioning and configuring
Monitoring is an important link in a DevOps toolchain. It allows IT organization to identify specific issues of specific releases and to understand the impact on end-users.[8] A summary of Monitor related activities are:
Information from monitoring activities often impacts Plan activities required for changes and for new release cycles.
Version Control
Version Control is an important link in a DevOps toolchain and a component of software configuration management. Version Control is the management of changes to documents, computer programs, large web sites, and other collections of information.[8] A summary of Version Control related activities are:
Non-linear development
Distributed development
Compatibility with existent systems and protocols
Toolkit-based design
Information from Version Control often supports Release activities required for changes and for new release cycles.
^ Garner Market Trends: DevOps – Not a Market, but Tool-Centric Philosophy That supports a Continuous Delivery Value Chain (Report). Gartner. 18 February 2015.
^ abcdefg Avoid Failure by Developing a Toolchain that Enables DevOps (Report). Gartner. 16 March 2016.
^ Best Practices in Change, Configuration and Release Management (Report). Gartner. 14 July 2010.
^ Roger S. Pressman (2009). Software Engineering: A Practitioner’s Approach (7th International ed.). New York: McGraw-Hill.
GitHub offers its basic services free of charge. Its more advanced professional and enterprise services are commercial.[5] Free GitHub accounts are commonly used to host open-source projects.[6] As of January 2019, GitHub offers unlimited private repositories to all plans, including free accounts, but allowed only up to three collaborators per repository for free.[7] Starting from April 15, 2020, the free plan allows unlimited collaborators, but restricts private repositories to 2,000 minutes of GitHub Actions[8] per month.[9] As of January 2020, GitHub reports having over 40 million users[10] and more than 190 million repositories[11] (including at least 28 million public repositories),[12] making it the largest host of source code in the world.[13]
History
GitHub at AWS Summit
The GitHub service was developed by Chris Wanstrath, P. J. Hyett, Tom Preston-Werner and Scott Chacon using Ruby on Rails, and started in February 2008. The company, GitHub, Inc., has existed since 2007 and is located in San Francisco.[14]The shading of the map illustrates the number of users as a proportion of each country’s Internet population. The circular charts surrounding the two hemispheres depict the total number of GitHub users (left) and commits (right) per country.
On February 24, 2009, GitHub announced that within the first year of being online, GitHub had accumulated over 46,000 public repositories, 17,000 of which were formed in the previous month. At that time, about 6,200 repositories had been forked at least once and 4,600 had been merged.
That same year, the site was harnessed by over 100,000 users, according to Github, and had grown to host 90,000 unique public repositories, 12,000 having been forked at least once, for a total of 135,000 repositories.[15]
In 2010, GitHub was hosting 1 million repositories.[16] A year later, this number doubled.[17]ReadWriteWeb reported that GitHub had surpassed SourceForge and Google Code in total number of commits for the period of January to May 2011.[18] On January 16, 2013, GitHub passed the 3 million users mark and was then hosting more than 5 million repositories.[19] By the end of the year, the number of repositories were twice as much, reaching 10 million repositories.[20]
In 2012, GitHub raised $100 million in funding from Andreessen Horowitz with $750 million valuation.[21]Peter Levine, general partner at Andreessen Horowitz, stated that GitHub had been growing revenue at 300% annually since 2008 “profitably nearly the entire way”.[22] On July 29, 2015, GitHub stated it had raised $250 million in funding in a round led by Sequoia Capital. Other investors of that round included Andreessen Horowitz, Thrive Capital, and IVP (Institutional Venture Partners).[23] The round valued the company at approximately $2 billion.[24]
In 2015, GitHub opened an office in Japan that is its first office outside of the U.S.[25] In 2016, GitHub was ranked No. 14 on the Forbes Cloud 100 list.[26] It has not been featured on the 2018, 2019 and 2020 lists.[27]
On February 28, 2018, GitHub fell victim to the third largest distributed denial-of-service (DDoS) attack in history, with incoming traffic reaching a peak of about 1.35 terabits per second.[28]
On June 19, 2018, GitHub expanded its GitHub Education by offering free education bundles to all schools.[29][30]
On June 4, 2018, Microsoft announced its intent to acquire GitHub for US$7.5 billion. The deal closed on October 26, 2018.[33] GitHub continued to operate independently as a community, platform and business.[34] Under Microsoft, the service was led by Xamarin‘s Nat Friedman, reporting to Scott Guthrie, executive vice president of Microsoft Cloud and AI. GitHub’s CEO, Chris Wanstrath, was retained as a “technical fellow”, also reporting to Guthrie.
There have been concerns from developers Kyle Simpson, JavaScript trainer and author, and Rafael Laguna, CEO at Open-Xchange over Microsoft’s purchase, citing uneasiness over Microsoft’s handling of previous acquisitions, such as Nokia’s mobile business or Skype.[35][36]
This acquisition was in line with Microsoft’s business strategy under CEO Satya Nadella, which has seen a larger focus on the cloud computing services, alongside development of and contributions to open-source software.[37][4][32]Harvard Business Review argued that Microsoft was intending to acquire GitHub to get access to its user base, so it can be used as a loss leader to encourage use of its other development products and services.[38]
Concerns over the sale bolstered interest in competitors: Bitbucket (owned by Atlassian), GitLab (a commercial open source product that also runs a hosted service version) and SourceForge (owned by BIZX, LLC) reported that they had seen spikes in new users intending to migrate projects from GitHub to their respective services.[39][40][41][42]
In September 2019, GitHub acquired Semmle, a code analysis tool.[43] In February 2020, GitHub launched in India under the name GitHub India Private Limited.[44] In March 2020, GitHub announced that they were acquiring npm, a JavaScript packaging vendor, for an undisclosed sum of money.[45] The deal was closed on 15 April 2020.[46]
In early July 2020, the GitHub Archive Program was established, to archive its open source code in perpetuity.[47]
Services
GitHub.com
Development of the GitHub.com platform began on October 19, 2007.[60][61][62] The site was launched in April 2008 by Tom Preston-Werner, Chris Wanstrath, P. J. Hyett and Scott Chacon after it had been made available for a few months prior as a beta release.[63]
Projects on GitHub.com can be accessed and managed using the standard Git command-line interface; all standard Git commands work with it. GitHub.com also allows users to browse public repositories on the site. Multiple desktop clients and Git plugins are also available. The site provides social networking-like functions such as feeds, followers, wikis (using wiki software called Gollum) and a social network graph to display how developers work on their versions (“forks“) of a repository and what fork (and branch within that fork) is newest.
Anyone can browse and download public repositories but only registered users can contribute content to repositories. With a registered user account, users are able to have discussions, manage repositories, submit contributions to others’ repositories, and review changes to code. GitHub.com began offering unlimited private repositories at no cost in January 2019 (limited to three contributors per project). Previously, only public repositories were free.[64][65][66] On April 14, 2020, GitHub made “all of the core GitHub features” free for everyone, including “private repositories with unlimited collaborators”.[67]
The fundamental software that underpins GitHub is Git itself, written by Linus Torvalds, creator of Linux. The additional software that provides the GitHub user interface was written using Ruby on Rails and Erlang by GitHub, Inc. developers Wanstrath,[68] Hyett, and Preston-Werner.
Scope
The main purpose of GitHub.com is to facilitate the version control and issue tracking aspects of software development. Labels, milestones, responsibility assignment, and a search engine are available for issue tracking. For version control, Git (and by extension GitHub.com) allows pull requests to propose changes to the source code. Users with the ability to review the proposed changes can see a diff of the requested changes and approve them. In Git terminology, this action is called “committing” and one instance of it is a “commit”. A history of all commits are kept and can be viewed at a later time.
In addition, GitHub supports the following formats and features:
Documentation, including automatically rendered README files in a variety of Markdown-like file formats (see README § On GitHub)
GitHub Actions, which allows building continuous integration and continuous deployment pipelines for testing, releasing and deploying software without the use of third-party websites/platforms
Graphs: pulse, contributors, commits, code frequency, punch card, network, members
3D render files that can be previewed using a new integrated STL file viewer that displays the files on a “3D canvas”.[72] The viewer is powered by WebGL and Three.js.
Photoshop’s native PSD format can be previewed and compared to previous versions of the same file.
GitHub’s Terms of Service do not require public software projects hosted on GitHub to meet the Open Source Definition. The terms of service state, “By setting your repositories to be viewed publicly, you agree to allow others to view and fork your repositories.”[73]
GitHub Enterprise
GitHub Enterprise is a self-managed version of GitHub.com with similar functionality. It can be run on an organization’s own hardware or on a cloud provider, and it has been available since November 2011.[74] In November 2020, source code for GitHub Enterprise Server was leaked online in apparent protest against DMCA takedown of YouTube-dl. According to GitHub, the source code came from GitHub accidentally sharing the code with Enterprise customers themselves, not from an attack on GitHub servers.[75][76]
GitHub Pages
GitHub Pages is a staticweb hosting service offered by GitHub since 2008 to GitHub users for hosting user blogs, project documentation,[77][78] or even whole books created as a page.[79]
All GitHub Pages content is stored in a Git repository, either as files served to visitors verbatim or in Markdown format. GitHub is seamlessly integrated with Jekyll static web site and blog generator and GitHub continuous integration pipelines. Each time the content source is updated, Jekyll regenerates the website and automatically serves it via GitHub Pages infrastructure.[80]
As with the rest of GitHub, it includes both free and paid tiers of service, instead of being supported by web advertising. Web sites generated through this service are hosted either as subdomains of the github.io domain, or as custom domains bought through a third-party domain name registrar.[81] When custom domain is set on a GitHub Pages repo a Let’s Encrypt certificate for it is generated automatically. Once the certificate has been generated Enforce HTTPS can be set for the repository’s website to transparently redirect all HTTP requests to HTTPS.[82][83]
Gist
GitHub also operates other services: a pastebin-style site called Gist[63] that is for hosting code snippets (GitHub proper is for hosting larger projects).
Tom Preston-Werner presented the then-new Gist feature at a punk rock Ruby conference in 2008.[84] Gist builds on the traditional simple concept of a pastebin by adding version control for code snippets, easy forking, and TLS encryption for private pastes. Because each “gist” has its own Git repository, multiple code snippets can be contained in a single paste and they can be pushed and pulled using Git. Further, forked code can be pushed back to the original author in the form of a patch, so gists (pastes) can become more like mini-projects.[citation needed]
Before February 18, 2018, unregistered users were able to upload text to the site. Since then, uploading gists has been deactivated for unregistered users with the aim to mitigate spamming.[85]
In 2016 GitHub announced the launch of the GitHub Campus Experts program[87] to train and encourage students to grow technology communities at their universities. The Campus Experts program is open to university students of 18 years and older across the world.[88] GitHub Campus Experts are one of the primary ways that GitHub funds student oriented events and communities, Campus Experts are given access to training, funding, and additional resources to run events and grow their communities. To become a Campus Expert applicants must complete an online training course consisting of multiple modules designed to grow community leadership skills.
GitHub Marketplace service
GitHub also provides some software as a service integrations for adding extra features to projects. Those services include:
Waffle.io: Project management for software teams. Automatically see pull requests, automated builds, reviews, and deployments across all of your repositories in GitHub.
Rollbar: Integrate with GitHub to provide real time debugging tools and full-stack exception reporting. It is compatible with all well used code languages, such as JavaScript, Python, .NET, Ruby, PHP, Node.js, Android, iOS, Go, Java, and C#.
Travis CI: To provide confidence for your apps while doing test and ship. Also gives full control over the build environment, to adapt it to the code. Supported languages: Go, Java, JavaScript, Objective-C, Python, PHP, Ruby, and Swift.
GitLocalize: Developed for teams that are translating their content from one point to another. GitLocalize automatically syncs with your repository so you can keep your workflow on GitHub. It also keeps you updated on what needs to be translated.
GitHub Sponsors
GitHub Sponsors allows users to make monthly money donations to projects hosted on GitHub.[89] The public beta was announced on May 23, 2019 and currently the project accepts wait list registrations. The Verge said that GitHub Sponsors “works exactly like Patreon” because “developers can offer various funding tiers that come with different perks, and they’ll receive recurring payments from supporters who want to access them and encourage their work” except with “zero fees to use the program”. Furthermore, GitHub offer incentives for early adopters during the first year: it pledges to cover payment processing costs, and match sponsorship payments up to $5,000 per developer. Furthermore, users still can use other similar services like Patreon and Open Collective and link to their own websites.[90][91]
GitHub Archive Program
In July 2020, GitHub stored a February archive of the site[47] in an abandoned mountain mine in Svalbard, Norway, part of the Arctic World Archive and not far from the Svalbard Global Seed Vault. The archive contained the code of all active public repositories, as well as that of dormant, but significant public repositories. The 21TB of data was stored on piqlFilm archival film reels as QR codes, and is expected to last 500–1,000 years.[92][93][94][95]
The GitHub Archive Program is also working with partners on Project Silica, in an attempt to store all public repositories for 10,000 years. It aims to write archives into the molecular structure of quartz glass platters, using a high-precision laser that pulses a quadrillion (1,000,000,000,000,000) times per second.[95]
Developed projects
Atom, a free and open-source text and source code editor
Electron, an open-source framework to use JavaScript-based websites as desktop applications.
Prominent users
Some prominent open source organizations and projects use GitHub as a primary place for collaboration, including:
A package manager or package-management system is a collection of software tools that automates the process of installing, upgrading, configuring, and removing computer programs for a computer‘s operating system in a consistent manner.[1]
A package manager deals with packages, distributions of software and data in archive files. Packages contain metadata, such as the software’s name, description of its purpose, version number, vendor, checksum (preferably a cryptographic hash function), and a list of dependencies necessary for the software to run properly. Upon installation, metadata is stored in a local package database. Package managers typically maintain a database of software dependencies and version information to prevent software mismatches and missing prerequisites. They work closely with software repositories, binary repository managers, and app stores.
Package managers are designed to eliminate the need for manual installs and updates. This can be particularly useful for large enterprises whose operating systems are typically consisting of hundreds or even tens of thousands of distinct software packages.[2]
Functions
Illustration of a package manager being used to download new software. Manual actions can include accepting a license agreement or selecting some package-specific configuration options.
A software package is an archive file containing a computer program as well as necessary metadata for its deployment. The computer program can be in source code that has to be compiled and built first.[3] Package metadata include package description, package version, and dependencies (other packages that need to be installed beforehand).
Package managers are charged with the task of finding, installing, maintaining or uninstalling software packages upon the user’s command. Typical functions of a package management system include:
Grouping packages by function to reduce user confusion
Managing dependencies to ensure a package is installed with all packages it requires, thus avoiding “dependency hell“
Challenges with shared libraries
Computer systems that rely on dynamic library linking, instead of static library linking, share executable libraries of machine instructions across packages and applications. In these systems, complex relationships between different packages requiring different versions of libraries results in a challenge colloquially known as “dependency hell“. On Microsoft Windows systems, this is also called “DLL hell” when working with dynamically linked libraries. Good package management is vital on these systems.[4] The Framework system from OPENSTEP was an attempt at solving this issue, by allowing multiple versions of libraries to be installed simultaneously, and for software packages to specify which version they were linked against.
Front-ends for locally compiled packages
System administrators may install and maintain software using tools other than package management software. For example, a local administrator may download unpackaged source code, compile it, and install it. This may cause the state of the local system to fall out of synchronization with the state of the package manager’s database. The local administrator will be required to take additional measures, such as manually managing some dependencies or integrating the changes into the package manager.
There are tools available to ensure that locally compiled packages are integrated with the package management. For distributions based on .deb and .rpm files as well as Slackware Linux, there is CheckInstall, and for recipe-based systems such as Gentoo Linux and hybrid systems such as Arch Linux, it is possible to write a recipe first, which then ensures that the package fits into the local package database.[citation needed]
Maintenance of configuration
Particularly troublesome with software upgrades are upgrades of configuration files. Since package managers, at least on Unix systems, originated as extensions of file archiving utilities, they can usually only either overwrite or retain configuration files, rather than applying rules to them. There are exceptions to this that usually apply to kernel configuration (which, if broken, will render the computer unusable after a restart). Problems can be caused if the format of configuration files changes; for instance, if the old configuration file does not explicitly disable new options that should be disabled. Some package managers, such as Debian‘s dpkg, allow configuration during installation. In other situations, it is desirable to install packages with the default configuration and then overwrite this configuration, for instance, in headless installations to a large number of computers. This kind of pre-configured installation is also supported by dpkg.
Repositories
To give users more control over the kinds of software that they are allowing to be installed on their system (and sometimes due to legal or convenience reasons on the distributors’ side), software is often downloaded from a number of software repositories.[5]
Upgrade suppression
When a user interacts with the package management software to bring about an upgrade, it is customary to present the user with the list of actions to be executed (usually the list of packages to be upgraded, and possibly giving the old and new version numbers), and allow the user to either accept the upgrade in bulk, or select individual packages for upgrades. Many package managers can be configured to never upgrade certain packages, or to upgrade them only when critical vulnerabilities or instabilities are found in the previous version, as defined by the packager of the software. This process is sometimes called version pinning.
For instance:
yum supports this with the syntax exclude=openoffice*[6]
pacman with IgnorePkg= openoffice[7] (to suppress upgrading openoffice in both cases)
dpkg and dselect support this partially through the hold flag in package selections
APT extends the hold flag through the complex “pinning” mechanism[8] (Users can also blacklist a package[9])
portage supports this through the package.mask configuration file
Cascading package removal
Some of the more advanced package management features offer “cascading package removal”,[7] in which all packages that depend on the target package and all packages that only the target package depends on, are also removed.
Comparison of commands
Although the commands are specific for every particular package manager, they are to a large extent translatable, as most package managers offer similar functions.
emerge -avtuDN --with-bdeps=y @world or emerge --update --pretend @world
delete orphans+config
zypper rm -u
pacman -Rsn $(pacman -Qdtq)
apt autoremove
dnf erase PKG
emerge --depclean
show orphans
zypper pa --orphaned --unneeded
pacman -Qdt
package-cleanup --quiet --leaves --exclude-bin
emerge -caD or emerge --depclean --pretend
update all
zypper up
pacman -Syu
apt upgrade
dnf update
emerge --update --deep --with-bdeps=y @world
The Arch Linux Pacman/Rosetta wiki offers an extensive overview.[11]
Prevalence
Package managers like dpkg have existed as early as 1994.[12]
Linux distributions oriented to binary packages rely heavily on package management systems as their primary means of managing and maintaining software. Mobile operating systems such as Android (Linux-based), iOS (Unix-like), and Windows Phone rely almost exclusively on their respective vendors’ app stores and thus use their own dedicated package management systems.
A package manager is often called an “install manager”, which can lead to a confusion between package managers and installers. The differences include:This box:
It is entirely at the discretion of the installer. It could be a file within the app’s folder, or among the operating system’s files and folders. At best, they may register themselves with an uninstallers list without exposing installation information.
There could be as many formats as the number of apps
Package format compatibility
Can be consumed as long as the package manager supports it. Either newer versions of the package manager keep supporting it or the user does not upgrade the package manager.
The installer is always compatible with its archive format, if it uses any. However, installers, like all computer programs, may be affected by software rot.
Comparison with build automation utility
Most software configuration management systems treat building software and deploying software as separate, independent steps. A build automation utility typically takes human-readable source code files already on a computer, and automates the process of converting them into a binary executable package on the same computer. Later a package manager typically running on some other computer downloads those pre-built binary executable packages over the internet and installs them.
However, both kinds of tools have many commonalities:
For example, the dependency graphtopological sorting used in a package manager to handle dependencies between binary components is also used in a build manager to handle the dependency between source components.
For example, many makefiles support not only building executables, but also installing them with make install.
For example, every package manager for a source-based distribution – Portage, Sorcery, Homebrew, etc. – supports converting human-readable source code to binary executables and installing it.
A few tools, such as Maak and A-A-P, are designed to handle both building and deployment, and can be used as either a build automation utility or as a package manager or both.[13]
Common package managers and formats
Universal package manager
Also known as binary repository manager, it is a software tool designed to optimize the download and storage of binary files, artifacts and packages used and produced in the software development process.[14] These package managers aim to standardize the way enterprises treat all package types. They give users the ability to apply security and compliance metrics across all artifact types. Universal package managers have been referred to as being at the center of a DevOps toolchain.[15]
Each package manager relies on the format and metadata of the packages it can manage. That is, package managers need groups of files to be bundled for the specific package manager along with appropriate metadata, such as dependencies. Often, a core set of utilities manages the basic installation from these packages and multiple package managers use these utilities to provide additional functionality.
For example, yum relies on rpm as a backend. Yum extends the functionality of the backend by adding features such as simple configuration for maintaining a network of systems. As another example, the Synaptic Package Manager provides a graphical user interface by using the Advanced Packaging Tool (apt) library, which, in turn, relies on dpkg for core functionality.
By the nature of free and open source software, packages under similar and compatible licenses are available for use on a number of operating systems. These packages can be combined and distributed using configurable and internally complex packaging systems to handle many permutations of software and manage version-specific dependencies and conflicts. Some packaging systems of free and open source software are also themselves released as free and open source software. One typical difference between package management in proprietary operating systems, such as Mac OS X and Windows, and those in free and open source software, such as Linux, is that free and open source software systems permit third-party packages to also be installed and upgraded through the same mechanism, whereas the package managers of Mac OS X and Windows will only upgrade software provided by Apple and Microsoft, respectively (with the exception of some third party drivers in Windows). The ability to continuously upgrade third party software is typically added by adding the URL of the corresponding repository to the package management’s configuration file.
Beside the system-level application managers, there are some add-on package managers for operating systems with limited capabilities and for programming languages in which developers need the latest libraries.
In contrast to system-level package managers, application-level package managers focus on a small part of the software system. They typically reside within a directory tree that is not maintained by the system-level package manager, such as c:\cygwin or /usr/local/fink. However, this might not be the case for the package managers that deal with programming libraries, leading to a possible conflict as both package managers may claim to “own” a file and might break upgrades.
Impact
Ian Murdock had commented that package management is “the single biggest advancement Linux has brought to the industry”, that it blurs the boundaries between operating system and applications, and that it makes it “easier to push new innovations […] into the marketplace and […] evolve the OS”.[16]
