Tools and processes are important, but it is more important to have competent people working together effectively.
Good documentation is useful in helping people to understand how the software is built and how to use it, but the main point of development is to create software, not documentation.
A contract is important but is no substitute for working closely with customers to discover what they need.
A project plan is important, but it must not be too rigid to accommodate changes in technology or the environment, stakeholders’ priorities, and people’s understanding of the problem and its solution.
Some of the authors formed the Agile Alliance, a non-profit organization that promotes software development according to the manifesto’s values and principles. Introducing the manifesto on behalf of the Agile Alliance, Jim Highsmith said,
The Agile movement is not anti-methodology, in fact many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment. Those who would brand proponents of XP or SCRUM or any of the other Agile Methodologies as “hackers” are ignorant of both the methodologies and the original definition of the term hacker.— Jim Highsmith, History: The Agile Manifesto
Agile software development principles
The Manifesto for Agile Software Development is based on twelve principles:” (WP)
Customer satisfaction by early and continuous delivery of valuable software.
Welcome changing requirements, even in late development.
Deliver working software frequently (weeks rather than months)
Close, daily cooperation between business people and developers
Projects are built around motivated individuals, who should be trusted
Face-to-face conversation is the best form of communication (co-location)
Working software is the primary measure of progress
Sustainable development, able to maintain a constant pace
Continuous attention to technical excellence and good design
Simplicity—the art of maximizing the amount of work not done—is essential
Best architectures, requirements, and designs emerge from self-organizing teams
Regularly, the team reflects on how to become more effective, and adjusts accordingly
Since then, software engineering undergraduate degrees have been established at many universities. A standard international curriculum for undergraduate software engineering degrees, SE2004, was defined by a steering committee between 2001 and 2004 with funding from the Association for Computing Machinery and the IEEE Computer Society. As of 2004, in the U.S., about 50 universities offer software engineering degrees, which teach both computer science and engineering principles and practices. The first software engineering Master’s degree was established at Seattle University in 1979. Since then graduate software engineering degrees have been made available from many more universities. Likewise in Canada, the Canadian Engineering Accreditation Board (CEAB) of the Canadian Council of Professional Engineers has recognized several software engineering programs.
In 1998, the US Naval Postgraduate School (NPS) established the first doctorate program in Software Engineering in the world. Additionally, many online advanced degrees in Software Engineering have appeared such as the Master of Science in Software Engineering (MSE) degree offered through the Computer Science and Engineering Department at California State University, Fullerton. Steve McConnell opines that because most universities teach computer science rather than software engineering, there is a shortage of true software engineers.ETS (École de technologie supérieure) University and UQAM (Université du Québec à Montréal) were mandated by IEEE to develop the Software Engineering Body of Knowledge (SWEBOK), which has become an ISO standard describing the body of knowledge covered by a software engineer.
Most software engineers work as employees or contractors. Software engineers work with businesses, government agencies (civilian or military), and non-profit organizations. Some software engineers work on their own as consulting software engineers. Some organizations have specialists to perform all of the tasks in the software development process. Other organizations separate software engineers based on specific software-engineering tasks. These companies sometimes hire interns (possibly university or college students) over a short time. In large projects, software engineers are distinguished from people who specialize in only one role because they take part in the design as well as the programming of the project. In small projects, software engineers will usually fill several or all roles at the same time. Specializations include:
Most students in the developed world have avoided degrees related to software engineering because of the fear of offshore outsourcing (importing software products or services from other countries) and of being displaced by foreign visa workers. Although government statistics do not currently show a threat to software engineering itself; a related career, computer programming does appear to have been affected. Often one is expected to start out as a computer programmer before being promoted to software engineer. Thus, the career path to software engineering may be rough, especially during recessions.
Some career counselors suggest a student also focus on “people skills” and business skills rather than purely technical skills because such “soft skills” are allegedly more difficult to offshore. Reasonable command over reading, writing & speaking English is asked by most of employers. It is the quasi-management aspects of software engineering that appear to be what has kept it from being impacted by globalization.
There are several prizes in the field of software engineering:
The Codie awards is a yearly award issued by the Software and Information Industry Association for excellence in software development within the software industry.
Margaret Hamilton promoted the term “software engineering” during her work on the Apollo program. The term “engineering” was used to acknowledge that the work should be taken just as seriously as other contributions toward the advancement of technology. Hamilton details her use of the term:
When I first came up with the term, no one had heard of it before, at least in our world. It was an ongoing joke for a long time. They liked to kid me about my radical ideas. It was a memorable day when one of the most respected hardware gurus explained to everyone in a meeting that he agreed with me that the process of building software should also be considered an engineering discipline, just like with hardware. Not because of his acceptance of the new “term” per se, but because we had earned his and the acceptance of the others in the room as being in an engineering field in its own right.
Suitability of the term
In each of the last few decades, at least one radical new approach has entered the mainstream of software development (e.g. Structured Programming, Object Orientation), implying that the field is still changing too rapidly to be considered an engineering discipline. Proponents argue that the supposedly radical new approaches are evolutionary rather than revolutionary.
Individual commentators have disagreed sharply on how to define software engineering or its legitimacy as an engineering discipline. David Parnas has said that software engineering is, in fact, a form of engineering.Steve McConnell has said that it is not, but that it should be.Donald Knuth has said that programming is an art and a science.Edsger W. Dijkstra claimed that the terms software engineering and software engineer have been misused[improper synthesis?] and should be considered harmful, particularly in the United States.
In Canada the use of the job title Engineer is controlled in each province by self-regulating professional engineering organizations who are also tasked with enforcement of the governing legislation. The intent is that any individual holding themselves out as an engineer has been verified to have been educated to a certain accredited level and their professional practice is subject to a code of ethics and peer scrutiny. It is also illegal to use the title Engineer in Canada unless an individual is licensed.
In Ontario, the Professional Engineers Act stipulates a minimum education level of a three-year diploma in technology from a College of Applied Arts and Technology or a degree in a relevant science area. However, engineering undergraduates and all other applicants are not allowed to use the title of engineer until they complete the minimum amount of work experience of four years in addition to completing the Professional Practice Examination (PPE). If the applicant does not hold an undergraduate engineering degree then they may have to take the Confirmatory Practice Exam or Specific Examination Program unless the exam requirements are waived by a committee.
IT professionals with degrees in other fields (such as computer science or information systems) are restricted from using the title Software Engineer, or wording Software Engineer in a title, depending on their province or territory of residence.
In some instances, cases have been taken to court regarding the illegal use of the protected title Engineer.
Throughout the whole of Europe, suitably qualified engineers may obtain the professional European Engineer qualification.
In France, the term ingénieur (engineer) is not a protected title and can be used by anyone, even by those who do not possess an academic degree.
However, the title Ingénieur Diplomé (Graduate Engineer) is an official academic title that is protected by the government and is associated with the Diplôme d’Ingénieur, which is one of the most prestigious academic degrees in France.
The use of the title tölvunarfræðingur (computer scientist) is protected by law in Iceland. Software engineering is taught in Computer Science departments in Icelandic universities. Icelandic law state that a permission must be obtained from the Minister of Industry when the degree was awarded abroad, prior to use of the title. The title is awarded to those who have obtained a BSc degree in Computer Science from a recognized higher educational institution.
In New Zealand, the Institution of Professional Engineers New Zealand (IPENZ), which licenses and regulates the country’s chartered engineers (CPEng), recognizes software engineering as a legitimate branch of professional engineering and accepts application of software engineers to obtain chartered status provided they have a tertiary degree of approved subjects. Software Engineering is included whereas Computer Science is normally not.
The Bureau of Labor Statistics (BLS) classifies computer software engineers as a subcategory of “computer specialists”, along with occupations such as computer scientist, Programmer, Database administrator and Network administrator. The BLS classifies all other engineering disciplines, including computer hardware engineers, as engineers.
Many states prohibit unlicensed persons from calling themselves an Engineer, or from indicating branches or specialties not covered licensing acts. In many states, the title Engineer is reserved for individuals with a Professional Engineering license indicating that they have shown minimum level of competency through accredited engineering education, qualified engineering experience, and engineering board’s examinations.
^Dijkstra, Edsger W; transcribed by Mario Béland (November 23, 2004) [First published December 3, 1993]. “There is still a war going on (manuscript Austin, 3 December 1993)”. E. W. Dijkstra Archive. The University of Texas at Austin, Department of Computer Sciences. Retrieved February 17, 2007. When the term was coined in 1968 by F.L. Bauer of the Technological University of Munich, I welcomed it. [. . .] I interpreted the introduction of the term “software engineering” as an apt reflection of the fact that the design of software systems was an activity par excellence for the mathematical engineer. [. . .]. As soon the term arrived in the USA, it was relieved of all its technical content. It had to be so for in its original meaning it was totally unacceptable [. . .] In the meantime, software engineering has become an almost empty term, as was nicely demonstrated by Data General who overnight promoted all its programmers to the exalted rank of “software engineer”!
Manning publishes the best quality IT books in the industry.
Manning is an independent publisher, providing computer books for software developers, engineers, architects, system administrators, and managers. Our books also cover topics for young programmers, students, and occasionally children.
Manning is an independent publisher of computer books and video courses for software developers, engineers, architects, system administrators, managers and all who are professionally involved with the computer business. We also publish for students and young programmers, including occasionally for children. We are an entirely virtual organization based on Shelter Island, New York, with many staff working from far-flung places like Manila and Zagreb.
“Independent” means we are not owned by a large corporate entity and are free to make decisions without bureaucratic overhead. That has allowed us to innovate and be flexible and to quickly adjust what we do as we go. We were the first by several years to sell our books as unprotected PDFs, something that later became commonplace. We were the first to start selling books before they were finished, in the Manning Early Access Program. This gave our readers access to our content as soon as it was readable, and this too has become common in the industry. And it means we are thinking every day about new ways to satisfy our customers, some of which we hope you will be pleased to discover in the not-too-distant future.
how we improve
We published our first book in 1993 and have been learning from our successes, and even more from our mistakes, ever since. Every new book teaches us something that helps us improve:
How to choose the topics we publish on
How to find the right authors for each book
How to help authors write the best books they can
How to ensure the content is valuable and easy to learn
How to let readers know about our content
We publish standalone titles as well as the following book series:
In a Month of Lunches
Readers can access our books through the Manning Early Access Program, O’Reilly Learning (formerly Safari Books Online), and iBooks. Print copies, wherever they are bought, come with free electronic versions in PDF, ePub and Kindle formats. With your print copy in hand, register it on the Manning site and you can download the digital versions from your account.
At this time, our eBooks are available only from Manning.com and Apple’s iBookstore.
Organizations today often struggle to balance business requirements with ever-increasing volumes of data. Additionally, the demand for leveraging large-scale, real-time data is growing rapidly among the most competitive digital industries. Conventional system architectures may not be up to the task. With this practical guide, you’ll learn how to leverage large-scale data usage across the business units in your organization using the principles of event-driven microservices.
Author Adam Bellemare takes you through the process of building an event-driven microservice-powered organization. You’ll reconsider how data is produced, accessed, and propagated across your organization. Learn powerful yet simple patterns for unlocking the value of this data. Incorporate event-driven design and architectural principles into your own systems. And completely rethink how your organization delivers value by unlocking near-real-time access to data at scale.
How to leverage event-driven architectures to deliver exceptional business value
The role of microservices in supporting event-driven designs
Architectural patterns to ensure success both within and between teams in your organization
Application patterns for developing powerful event-driven microservices
Components and tooling required to get your microservice ecosystem off the ground
About the Author:
Adam Bellemare is a Staff Engineer, Data Platform at Flipp. He’s held this position since 2017. He joined Flipp in 2014 as a senior developer at Flipp. Prior to that, he held positions in embedded software development and quality assurance. His expertise includes: Devops (Kafka, Spark, Mesos, Zookeeper Clusters. Programmatic Building, scaling, destroying); Technical Leadership (Bringing Avro formatting to our data end-to-end, championing Kafka as the event-driven microservice bus, prototyping JRuby, Scala and Java Kafka clients and focusing on removing technical impediments to allow for product delivery); Software Development (Building microservices in Java and Scala using Spark and Kafka libraries); and Data Engineering (Reshaping the way that behavioral data is collected from user devices and shared with our Machine Learning, Billing and Analytics teams).
Conventions Used in This Book
O’Reilly Online Learning
How to Contact Us
1. Why Event-Driven Microservices
What Are Event-Driven Microservices?
Introduction to Domain-Driven Design and Bounded Contexts
Leveraging Domain Models and Bounded Contexts
Aligning Bounded Contexts with Business Requirements
Business Communication Structures
Implementation Communication Structures
Data Communication Structures
Conway’s Law and Communication Structures
Communication Structures in Traditional Computing
Option 1: Make a New Service
Option 2: Add It to the Existing Service
Pros and Cons of Each Option
The Team Scenario, Continued
Event-Driven Communication Structures
Events Are the Basis of Communication
Event Streams Provide the Single Source of Truth
Consumers Perform Their Own Modeling and Querying
Data Communication Is Improved Across the Organization
Accessible Data Supports Business Communication Changes
Asynchronous Event-Driven Microservices
Example Team Using Event-Driven Microservices
Drawbacks of Synchronous Microservices
Benefits of Synchronous Microservices
2. Event-Driven Microservice Fundamentals
The Contents of an Event
The Structure of an Event
Materializing State from Entity Events
Event Data Definitions and Schemas
Microservice Single Writer Principle
Powering Microservices with the Event Broker
Event Storage and Serving
Additional Factors to Consider
Event Brokers Versus Message Brokers
Consuming from the Immutable Log
Providing a Single Source of Truth
Managing Microservices at Scale
Putting Microservices into Containers
Putting Microservices into Virtual Machines
Managing Containers and Virtual Machines
Paying the Microservice Tax
3. Communication and Data Contracts
Event-Driven Data Contracts
Using Explicit Schemas as Contracts
Schema Definition Comments
Full-Featured Schema Evolution
Code Generator Support
Breaking Schema Changes
Selecting an Event Format
Tell the Truth, the Whole Truth, and Nothing but the Truth
Use a Singular Event Definition per Stream
Use the Narrowest Data Types
Keep Events Single-Purpose
Minimize the Size of Events
Involve Prospective Consumers in the Event Design
Avoid Events as Semaphores or Signals
4. Integrating Event-Driven Architectures with Existing Systems
What Is Data Liberation?
Compromises for Data Liberation
Converting Liberated Data to Events
Data Liberation Patterns
Data Liberation Frameworks
Liberating Data by Query
Incremental Timestamp Loading
Autoincrementing ID Loading
Benefits of Query-Based Updating
Drawbacks of Query-Based Updating
Liberating Data Using Change-Data Capture Logs
Benefits of Using Data Store Logs
Drawbacks of Using Data Base Logs
Liberating Data Using Outbox Tables
Isolating Internal Data Models
Ensuring Schema Compatibility
Capturing Change-Data Using Triggers
Making Data Definition Changes to Data Sets Under Capture
Handling After-the-Fact Data Definition Changes for the Query and CDC Log Patterns
Handling Data Definition Changes for Change-Data Table Capture Patterns
Sinking Event Data to Data Stores
The Impacts of Sinking and Sourcing on a Business
5. Event-Driven Processing Basics
Composing Stateless Topologies
Branching and Merging Streams
Repartitioning Event Streams
Example: Repartitioning an Event Stream
Copartitioning Event Streams
Example: Copartitioning an Event Stream
Assigning Partitions to a Consumer Instance
Assigning Partitions with the Partition Assignor
Assigning Copartitioned Partitions
Partition Assignment Strategies
Recovering from Stateless Processing Instance Failures
6. Deterministic Stream Processing
Determinism with Event-Driven Workflows
Synchronizing Distributed Timestamps
Processing with Timestamped Events
Event Scheduling and Deterministic Processing
Custom Event Schedulers
Processing Based on Event Time, Processing Time, and Ingestion Time
Timestamp Extraction by the Consumer
Request-Response Calls to External Systems
Watermarks in Parallel Processing
Stream Time in Parallel Processing
Out-of-Order and Late-Arriving Events
Late Events with Watermarks and Stream Time
Causes and Impacts of Out-of-Order Events
Time-Sensitive Functions and Windowing
Handling Late Events
Reprocessing Versus Processing in Near-Real Time
Intermittent Failures and Late Events
Producer/Event Broker Connectivity Issues
Summary and Further Reading
7. Stateful Streaming
State Stores and Materializing State from an Event Stream
Recording State to a Changelog Event Stream
Materializing State to an Internal State Store
Materializing Global State
Advantages of Using Internal State
Disadvantages of Using Internal State
Scaling and Recovery of Internal State
Materializing State to an External State Store
Advantages of External State
Drawbacks of External State
Scaling and Recovery with External State Stores
Rebuilding Versus Migrating State Stores
Transactions and Effectively Once Processing
Example: Stock Accounting Service
Effectively Once Processing with Client-Broker Transactions
Effectively Once Processing Without Client-Broker Transactions
8. Building Workflows with Microservices
The Choreography Pattern
A Simple Event-Driven Choreography Example
Creating and Modifying a Choreographed Workflow
Monitoring a Choreographed Workflow
The Orchestration Pattern
A Simple Event-Driven Orchestration Example
A Simple Direct-Call Orchestration Example
Comparing Direct-Call and Event-Driven Orchestration
Creating and Modifying an Orchestration Workflow
Monitoring the Orchestration Workflow
Choreographed Transactions: The Saga Pattern
9. Microservices Using Function-as-a-Service
Designing Function-Based Solutions as Microservices
Ensure Strict Membership to a Bounded Context
Commit Offsets Only After Processing Has Completed
Less Is More
Choosing a FaaS Provider
Building Microservices Out of Functions
Cold Start and Warm Starts
Starting Functions with Triggers
Triggering Based on New Events: The Event-Stream Listener
Triggering Based on Consumer Group Lag
Triggering on a Schedule
Triggering Using Webhooks
Triggering on Resource Events
Performing Business Work with Functions
Functions Calling Other Functions
Event-Driven Communication Pattern
Termination and Shutdown
Tuning Your Functions
Allocating Sufficient Resources
Batch Event-Processing Parameters
Scaling Your FaaS Solutions
10. Basic Producer and Consumer Microservices
Where Do BPCs Work Well?
Integration with Existing and Legacy Systems
Stateful Business Logic That Isn’t Reliant Upon Event Order
When the Data Layer Does Much of the Work
Independent Scaling of the Processing and Data Layer
Hybrid BPC Applications with External Stream Processing
Example: Using an External Stream-Processing Framework to Join Event Streams
11. Heavyweight Framework Microservices
A Brief History of Heavyweight Frameworks
The Inner Workings of Heavyweight Frameworks
Benefits and Limitations
Cluster Setup Options and Execution Modes
Use a Hosted Service
Build Your Own Full Cluster
Create Clusters with CMS Integration
Application Submission Modes
Handling State and Using Checkpoints
Scaling Applications and Handling Event Stream Partitions
Scaling an Application While It Is Running
Scaling an Application by Restarting It
Recovering from Failures
Languages and Syntax
Choosing a Framework
Example: Session Windowing of Clicks and Views
12. Lightweight Framework Microservices
Benefits and Limitations
Handling State and Using Changelogs
Scaling Applications and Recovering from Failures
State Replication and Hot Replicas
Choosing a Lightweight Framework
Apache Kafka Streams
Apache Samza: Embedded Mode
Languages and Syntax
Stream-Table-Table Join: Enrichment Pattern
13. Integrating Event-Driven and Request-Response Microservices
Handling External Events
Autonomously Generated Events
Reactively Generated Events
Handling Autonomously Generated Analytical Events
Integrating with Third-Party Request-Response APIs
Processing and Serving Stateful Data
Serving Real-Time Requests with Internal State Stores
Serving Real-Time Requests with External State Stores
Handling Requests Within an Event-Driven Workflow
Processing Events for User Interfaces
Micro-Frontends in Request-Response Applications
The Benefits of Microfrontends
Easy Alignment to Business Requirements
Drawbacks of Microfrontends
Potentially Inconsistent UI Elements and Styling
Varying Microfrontend Performance
Example: Experience Search and Review Application
14. Supportive Tooling
Microservice-to-Team Assignment System
Event Stream Creation and Modification
Event Stream Metadata Tagging
Schema Creation and Modification Notifications
Permissions and Access Control Lists for Event Streams
State Management and Application Reset
Consumer Offset Lag Monitoring
Streamlined Microservice Creation Process
Container Management Controls
Cluster Creation and Management
Programmatic Bringup of Event Brokers
Programmatic Bringup of Compute Resources
Cross-Cluster Event Data Replication
Programmatic Bringup of Tooling
Dependency Tracking and Topology Visualization
15. Testing Event-Driven Microservices
General Testing Principles
Unit-Testing Topology Functions
Testing the Topology
Testing Schema Evolution and Compatibility
Integration Testing of Event-Driven Microservices
Local Integration Testing
Create a Temporary Environment Within the Runtime of Your Test Code
Create a Temporary Environment External to Your Test Code
Integrate Hosted Services Using Mocking and Simulator Options
Integrate Remote Services That Have No Local Options
Full Remote Integration Testing
Programmatically Create a Temporary Integration Testing Environment
Testing Using a Shared Environment
Testing Using the Production Environment
Choosing Your Full-Remote Integration Testing Strategy
16. Deploying Event-Driven Microservices
Principles of Microservice Deployment
Architectural Components of Microservice Deployment
Continuous Integration, Delivery, and Deployment Systems
Container Management Systems and Commodity Hardware
The Basic Full-Stop Deployment Pattern
The Rolling Update Pattern
The Breaking Schema Change Pattern
Eventual Migration via Two Event Streams
Synchronized Migration to the New Event Stream
The Blue-Green Deployment Pattern
Business Domains and Bounded Contexts
Shareable Tools and Infrastructure
Data Liberation and the Single Source of Truth
Microservice Implementation Options
GitHub offers its basic services free of charge. Its more advanced professional and enterprise services are commercial. Free GitHub accounts are commonly used to host open-source projects. As of January 2019, GitHub offers unlimited private repositories to all plans, including free accounts, but allowed only up to three collaborators per repository for free. Starting from April 15, 2020, the free plan allows unlimited collaborators, but restricts private repositories to 2,000 minutes of GitHub Actions per month. As of January 2020, GitHub reports having over 40 million users and more than 190 million repositories (including at least 28 million public repositories), making it the largest host of source code in the world.
GitHub at AWS Summit
The GitHub service was developed by Chris Wanstrath, P. J. Hyett, Tom Preston-Werner and Scott Chacon using Ruby on Rails, and started in February 2008. The company, GitHub, Inc., has existed since 2007 and is located in San Francisco.The shading of the map illustrates the number of users as a proportion of each country’s Internet population. The circular charts surrounding the two hemispheres depict the total number of GitHub users (left) and commits (right) per country.
On February 24, 2009, GitHub announced that within the first year of being online, GitHub had accumulated over 46,000 public repositories, 17,000 of which were formed in the previous month. At that time, about 6,200 repositories had been forked at least once and 4,600 had been merged.
That same year, the site was harnessed by over 100,000 users, according to Github, and had grown to host 90,000 unique public repositories, 12,000 having been forked at least once, for a total of 135,000 repositories.
In 2010, GitHub was hosting 1 million repositories. A year later, this number doubled.ReadWriteWeb reported that GitHub had surpassed SourceForge and Google Code in total number of commits for the period of January to May 2011. On January 16, 2013, GitHub passed the 3 million users mark and was then hosting more than 5 million repositories. By the end of the year, the number of repositories were twice as much, reaching 10 million repositories.
In 2012, GitHub raised $100 million in funding from Andreessen Horowitz with $750 million valuation.Peter Levine, general partner at Andreessen Horowitz, stated that GitHub had been growing revenue at 300% annually since 2008 “profitably nearly the entire way”. On July 29, 2015, GitHub stated it had raised $250 million in funding in a round led by Sequoia Capital. Other investors of that round included Andreessen Horowitz, Thrive Capital, and IVP (Institutional Venture Partners). The round valued the company at approximately $2 billion.
In 2015, GitHub opened an office in Japan that is its first office outside of the U.S. In 2016, GitHub was ranked No. 14 on the Forbes Cloud 100 list. It has not been featured on the 2018, 2019 and 2020 lists.
On February 28, 2018, GitHub fell victim to the third largest distributed denial-of-service (DDoS) attack in history, with incoming traffic reaching a peak of about 1.35 terabits per second.
On June 19, 2018, GitHub expanded its GitHub Education by offering free education bundles to all schools.
On June 4, 2018, Microsoft announced its intent to acquire GitHub for US$7.5 billion. The deal closed on October 26, 2018. GitHub continued to operate independently as a community, platform and business. Under Microsoft, the service was led by Xamarin‘s Nat Friedman, reporting to Scott Guthrie, executive vice president of Microsoft Cloud and AI. GitHub’s CEO, Chris Wanstrath, was retained as a “technical fellow”, also reporting to Guthrie.
This acquisition was in line with Microsoft’s business strategy under CEO Satya Nadella, which has seen a larger focus on the cloud computing services, alongside development of and contributions to open-source software.Harvard Business Review argued that Microsoft was intending to acquire GitHub to get access to its user base, so it can be used as a loss leader to encourage use of its other development products and services.
In early July 2020, the GitHub Archive Program was established, to archive its open source code in perpetuity.
Development of the GitHub.com platform began on October 19, 2007. The site was launched in April 2008 by Tom Preston-Werner, Chris Wanstrath, P. J. Hyett and Scott Chacon after it had been made available for a few months prior as a beta release.
Projects on GitHub.com can be accessed and managed using the standard Git command-line interface; all standard Git commands work with it. GitHub.com also allows users to browse public repositories on the site. Multiple desktop clients and Git plugins are also available. The site provides social networking-like functions such as feeds, followers, wikis (using wiki software called Gollum) and a social network graph to display how developers work on their versions (“forks“) of a repository and what fork (and branch within that fork) is newest.
Anyone can browse and download public repositories but only registered users can contribute content to repositories. With a registered user account, users are able to have discussions, manage repositories, submit contributions to others’ repositories, and review changes to code. GitHub.com began offering unlimited private repositories at no cost in January 2019 (limited to three contributors per project). Previously, only public repositories were free. On April 14, 2020, GitHub made “all of the core GitHub features” free for everyone, including “private repositories with unlimited collaborators”.
The fundamental software that underpins GitHub is Git itself, written by Linus Torvalds, creator of Linux. The additional software that provides the GitHub user interface was written using Ruby on Rails and Erlang by GitHub, Inc. developers Wanstrath, Hyett, and Preston-Werner.
The main purpose of GitHub.com is to facilitate the version control and issue tracking aspects of software development. Labels, milestones, responsibility assignment, and a search engine are available for issue tracking. For version control, Git (and by extension GitHub.com) allows pull requests to propose changes to the source code. Users with the ability to review the proposed changes can see a diff of the requested changes and approve them. In Git terminology, this action is called “committing” and one instance of it is a “commit”. A history of all commits are kept and can be viewed at a later time.
In addition, GitHub supports the following formats and features:
GitHub’s Terms of Service do not require public software projects hosted on GitHub to meet the Open Source Definition. The terms of service state, “By setting your repositories to be viewed publicly, you agree to allow others to view and fork your repositories.”
GitHub Enterprise is a self-managed version of GitHub.com with similar functionality. It can be run on an organization’s own hardware or on a cloud provider, and it has been available since November 2011. In November 2020, source code for GitHub Enterprise Server was leaked online in apparent protest against DMCA takedown of YouTube-dl. According to GitHub, the source code came from GitHub accidentally sharing the code with Enterprise customers themselves, not from an attack on GitHub servers.
All GitHub Pages content is stored in a Git repository, either as files served to visitors verbatim or in Markdown format. GitHub is seamlessly integrated with Jekyll static web site and blog generator and GitHub continuous integration pipelines. Each time the content source is updated, Jekyll regenerates the website and automatically serves it via GitHub Pages infrastructure.
As with the rest of GitHub, it includes both free and paid tiers of service, instead of being supported by web advertising. Web sites generated through this service are hosted either as subdomains of the github.io domain, or as custom domains bought through a third-party domain name registrar. When custom domain is set on a GitHub Pages repo a Let’s Encrypt certificate for it is generated automatically. Once the certificate has been generated Enforce HTTPS can be set for the repository’s website to transparently redirect all HTTP requests to HTTPS.
GitHub also operates other services: a pastebin-style site called Gist that is for hosting code snippets (GitHub proper is for hosting larger projects).
Tom Preston-Werner presented the then-new Gist feature at a punk rock Ruby conference in 2008. Gist builds on the traditional simple concept of a pastebin by adding version control for code snippets, easy forking, and TLS encryption for private pastes. Because each “gist” has its own Git repository, multiple code snippets can be contained in a single paste and they can be pushed and pulled using Git. Further, forked code can be pushed back to the original author in the form of a patch, so gists (pastes) can become more like mini-projects.
Before February 18, 2018, unregistered users were able to upload text to the site. Since then, uploading gists has been deactivated for unregistered users with the aim to mitigate spamming.
In 2016 GitHub announced the launch of the GitHub Campus Experts program to train and encourage students to grow technology communities at their universities. The Campus Experts program is open to university students of 18 years and older across the world. GitHub Campus Experts are one of the primary ways that GitHub funds student oriented events and communities, Campus Experts are given access to training, funding, and additional resources to run events and grow their communities. To become a Campus Expert applicants must complete an online training course consisting of multiple modules designed to grow community leadership skills.
GitHub Marketplace service
GitHub also provides some software as a service integrations for adding extra features to projects. Those services include:
Waffle.io: Project management for software teams. Automatically see pull requests, automated builds, reviews, and deployments across all of your repositories in GitHub.
GitLocalize: Developed for teams that are translating their content from one point to another. GitLocalize automatically syncs with your repository so you can keep your workflow on GitHub. It also keeps you updated on what needs to be translated.
GitHub Sponsors allows users to make monthly money donations to projects hosted on GitHub. The public beta was announced on May 23, 2019 and currently the project accepts wait list registrations. The Verge said that GitHub Sponsors “works exactly like Patreon” because “developers can offer various funding tiers that come with different perks, and they’ll receive recurring payments from supporters who want to access them and encourage their work” except with “zero fees to use the program”. Furthermore, GitHub offer incentives for early adopters during the first year: it pledges to cover payment processing costs, and match sponsorship payments up to $5,000 per developer. Furthermore, users still can use other similar services like Patreon and Open Collective and link to their own websites.
GitHub Archive Program
In July 2020, GitHub stored a February archive of the site in an abandoned mountain mine in Svalbard, Norway, part of the Arctic World Archive and not far from the Svalbard Global Seed Vault. The archive contained the code of all active public repositories, as well as that of dormant, but significant public repositories. The 21TB of data was stored on piqlFilm archival film reels as QR codes, and is expected to last 500–1,000 years.
The GitHub Archive Program is also working with partners on Project Silica, in an attempt to store all public repositories for 10,000 years. It aims to write archives into the molecular structure of quartz glass platters, using a high-precision laser that pulses a quadrillion (1,000,000,000,000,000) times per second.
Atom, a free and open-source text and source code editor
Some prominent open source organizations and projects use GitHub as a primary place for collaboration, including:
As of February 2021, the platform has more than 40 million students, 155,000 courses and 70,000 instructors teaching courses in over 65 languages. There have been over 480 million course enrollments. Students and instructors come from 180+ countries and 2/3 of the students are located outside of the U.S.
Students take courses largely as a means of improving job-related skills. Some courses generate credit toward technical certification. Udemy has made a special effort to attract corporate trainers seeking to create coursework for employees of their company. As of 2021, there are more than 155,000 courses on the website.
The headquarters of Udemy is located in San Francisco, California, with offices in Denver, Colorado; Dublin, Ireland; Ankara, Turkey; Sao Paulo, Brazil; and Gurugram, India.
“It has been influential to the field of software engineering and is regarded as an important source for object-oriented design theory and practice. More than 500,000 copies have been sold in English and in 13 other languages. The authors are often referred to as the Gang of Four (GoF).” (WP)
Capturing a wealth of experience about the design of object-oriented software, four top-notch designers present a catalog of simple and succinct solutions to commonly occurring design problems. Previously undocumented, these 23 patterns allow designers to create more flexible, elegant, and ultimately reusable designs without having to rediscover the design solutions themselves.
The authors begin by describing what patterns are and how they can help you design object-oriented software. They then go on to systematically name, explain, evaluate, and catalog recurring designs in object-oriented systems. With Design Patterns as your guide, you will learn how these important patterns fit into the software development process, and how you can leverage them to solve your own design problems most efficiently.
Each pattern describes the circumstances in which it is applicable, when it can be applied in view of other design constraints, and the consequences and trade-offs of using the pattern within a larger design. All patterns are compiled from real systems and are based on real-world examples. Each pattern also includes code that demonstrates how it may be implemented in object-oriented programming languages like C++ or Smalltalk.
Design Patterns is a modern classic in the literature of object-oriented development, offering timeless and elegant solutions to common problems in software design. It describes patterns for managing object creation, composing objects into larger structures, and coordinating control flow between objects. The book provides numerous examples where using composition rather than inheritance can improve the reusability and flexibility of code. Note, though, that it’s not a tutorial but a catalog that you can use to find an object-oriented design pattern that’s appropriate for the needs of your particular application–a selection for virtuoso programmers who appreciate (or require) consistent, well-engineered object-oriented designs
You know you don’t want to reinvent the wheel, so you look to Design Patterns: the lessons learned by those who’ve faced the same software design problems. With Design Patterns, you get to take advantage of the best practices and experience of others so you can spend your time on something more challenging. Something more fun. This book shows you the patterns that matter, when to use them and why, how to apply them to your own designs, and the object-oriented design principles on which they’re based. Join hundreds of thousands of developers who’ve improved their object-oriented design skills through Head First Design Patterns.
What’s so special about this book?
If you’ve read a Head First book, you know what to expect: a visually rich format designed for the way your brain works. With Head First Design Patterns, 2E you’ll learn design principles and patterns in a way that won’t put you to sleep, so you can get out there to solve software design problems and speak the language of patterns with others on your team.
“To the Gang of Four, whose insight and expertise in capturing and communicating Design Patterns has changed the face of software design forever, and bettered the lives of developers throughout the world. But seriously, when are we going to see a second edition? After all, it’s been only twenty-five years.” (HFDP)
In software engineering, a software design pattern is a general, reusable solution to a commonly occurring problem within a given context in software design. It is not a finished design that can be transformed directly into source or machine code. Rather, it is a description or template for how to solve a problem that can be used in many different situations. Design patterns are formalized best practices that the programmer can use to solve common problems when designing an application or system.
Object-oriented design patterns typically show relationships and interactions between classes or objects, without specifying the final application classes or objects that are involved. Patterns that imply mutable state may be unsuited for functional programming languages, some patterns can be rendered unnecessary in languages that have built-in support for solving the problem they are trying to solve, and object-oriented patterns are not necessarily suitable for non-object-oriented languages.
In a recent review study, Wedyan and Abufakher investigate design patterns and software quality and conclude: “Our study has shown that the primary studies provide an empirical evidence on the positive effect of documentation of designs pattern instances on program comprehension, and therefore, maintainability. While this result is not surprising, it has, however, two indications. First, developers should pay more effort to add such documentation, even if in the form of simple comments in the source code. Second, when comparing results of different studies, the effect of documentation has to be considered.”
Kubuntu was sponsored by Canonical Ltd. until 2012 and then directly by Blue Systems. Now, employees of Blue Systems contribute upstream, to KDE and Debian, and Kubuntu development is led by community contributors. During the changeover, Kubuntu retained the use of Ubuntu project servers and existing developers.
Kubuntu was born on 10 December 2004 at the Ubuntu Mataro Conference in Mataró, Spain.
The Linux Mint project was created by Clément Lefèbvre and is actively maintained by the Linux Mint Team and community.
Linux Mint began in 2006 with a beta release, 1.0, code-named ‘Ada’, based on Kubuntu. Linux Mint 2.0 ‘Barbara’ was the first version to use Ubuntu as its codebase. It had few users until the release of Linux Mint 3.0, ‘Cassandra’.
Ubuntu is released every six months, with long-term support (LTS) releases every two years. As of 22 October 2020, the most recent long-term support release is 20.04 (“Focal Fossa”), which is supported until 2025 under public support and until 2030 as a paid option. The latest standard release is 20.10 (“Groovy Gorilla”), which is supported for nine months.
Ubuntu is developed by Canonical, and a community of other developers, under a meritocratic governance model. Canonical provides security updates and support for each Ubuntu release, starting from the release date and until the release reaches its designated end-of-life (EOL) date. Canonical generates revenue through the sale of premium services related to Ubuntu.
Ubuntu is named after the Nguni philosophy of ubuntu, which Canonical indicates means “humanity to others” with a connotation of “I am what I am because of who we all are”.