Categories
Bibliography DevOps DevSecOps-Security-Privacy Java Software Engineering Spring Framework

B081W4C2DH ISBN-13: 978-1484250518

See: Pro Spring Security: Securing Spring Framework 5 and Boot 2-based Java Applications, 2nd Edition, Publisher ‏ : ‎ Apress; 2nd ed. edition (November 22, 2019)

See also: Spring Bibliography, Spring Framework and Cloud Native

Fair Use Source:

Categories
Bibliography Go Programming Language Java Python Software Engineering

B0845YMM37

See: gRPC: Up and Running: Building Cloud Native Applications with Go and Java for Docker and Kubernetes 1st Edition

Fair Use Source:

Categories
Bibliography DevOps DevSecOps-Security-Privacy Software Engineering

B0872KBFF4

See: Distributed Tracing in Practice: Instrumenting, Analyzing, and Debugging Microservices 1st Edition

Fair Use Source:

Categories
Bibliography DevOps DevSecOps-Security-Privacy JavaScript Software Engineering

B07V78WH7V

See: Web Security for Developers: Real Threats, Practical Defense Illustrated Edition

Fair Use Source:

Categories
Azure Bibliography DevOps DevSecOps-Security-Privacy

B08F5JHZJG

See: Exam Ref AZ-500 Microsoft Azure Security Technologies

Fair Use Source:

Categories
Bibliography

B081TKSSNN

See: Monolith to Microservices: Evolutionary Patterns to Transform Your Monolith 1st Edition

Fair Use Source:

Categories
Bibliography

B08P8BS6BL

See: Traefik API Gateway for Microservices: With Java and Python Microservices Deployed in Kubernetes 1st ed. Edition

See also Kubernetes and Cloud Native

Fair Use Source:

Categories
Bibliography

1617295957

See: Microservices Security in Action: Design secure network and API endpoint security for Microservices applications, with examples using Java, Kubernetes, and Istio 1st Edition

See also Kubernetes and Cloud Native

Fair Use Source: https://www.manning.com/books/microservices-security-in-action

Categories
Bibliography DevOps Java Spring Framework

B094DMGZ6T ISBN-13: 978-1801072977

See: Microservices with Spring Boot and Spring Cloud: Build resilient and scalable microservices using Spring Cloud, Istio, and Kubernetes, 2nd Edition, Publisher ‏ : ‎ Packt Publishing; 2nd ed. edition (August 10, 2021)

See also Spring Framework, Kubernetes and Cloud Native

Fair Use Source:

Categories
Bibliography Cloud Data Science - Big Data DevOps Software Engineering

Building Event-Driven Microservices: Leveraging Organizational Data at Scale

See also: Event-driven architecture (EDA), Microservices, API, API Bibliography – Application Programming Interface Books and References

Building Event-Driven Microservices: Leveraging Organizational Data at Scale, 1st Edition, by Adam Bellemare, 2020, B08C9V1FC9 (BEDM)

Fair Use Source: B08C9V1FC9 (BEDM)

About This Book:

Organizations today often struggle to balance business requirements with ever-increasing volumes of data. Additionally, the demand for leveraging large-scale, real-time data is growing rapidly among the most competitive digital industries. Conventional system architectures may not be up to the task. With this practical guide, you’ll learn how to leverage large-scale data usage across the business units in your organization using the principles of event-driven microservices.

Author Adam Bellemare takes you through the process of building an event-driven microservice-powered organization. You’ll reconsider how data is produced, accessed, and propagated across your organization. Learn powerful yet simple patterns for unlocking the value of this data. Incorporate event-driven design and architectural principles into your own systems. And completely rethink how your organization delivers value by unlocking near-real-time access to data at scale.

You’ll learn:

  • How to leverage event-driven architectures to deliver exceptional business value
  • The role of microservices in supporting event-driven designs
  • Architectural patterns to ensure success both within and between teams in your organization
  • Application patterns for developing powerful event-driven microservices
  • Components and tooling required to get your microservice ecosystem off the ground

About the Author:

Adam Bellemare is a Staff Engineer, Data Platform at Flipp. He’s held this position since 2017. He joined Flipp in 2014 as a senior developer at Flipp. Prior to that, he held positions in embedded software development and quality assurance. His expertise includes: Devops (Kafka, Spark, Mesos, Zookeeper Clusters. Programmatic Building, scaling, destroying); Technical Leadership (Bringing Avro formatting to our data end-to-end, championing Kafka as the event-driven microservice bus, prototyping JRuby, Scala and Java Kafka clients and focusing on removing technical impediments to allow for product delivery); Software Development (Building microservices in Java and Scala using Spark and Kafka libraries); and Data Engineering (Reshaping the way that behavioral data is collected from user devices and shared with our Machine Learning, Billing and Analytics teams).

Book Details:

  • ASIN : B08C9V1FC9
  • Publisher : O’Reilly Media; 1st edition (July 2, 2020)
  • Publication date : July 2, 2020
  • Print length : 511 pages

Table of Contents:

  1. Preface
    1. Conventions Used in This Book
    2. O’Reilly Online Learning
    3. How to Contact Us
    4. Acknowledgments
  2. 1. Why Event-Driven Microservices
    1. What Are Event-Driven Microservices?
    2. Introduction to Domain-Driven Design and Bounded Contexts
      1. Leveraging Domain Models and Bounded Contexts
      2. Aligning Bounded Contexts with Business Requirements
    3. Communication Structures
      1. Business Communication Structures
      2. Implementation Communication Structures
      3. Data Communication Structures
      4. Conway’s Law and Communication Structures
    4. Communication Structures in Traditional Computing
      1. Option 1: Make a New Service
      2. Option 2: Add It to the Existing Service
      3. Pros and Cons of Each Option
      4. The Team Scenario, Continued
      5. Conflicting Pressures
    5. Event-Driven Communication Structures
      1. Events Are the Basis of Communication
      2. Event Streams Provide the Single Source of Truth
      3. Consumers Perform Their Own Modeling and Querying
      4. Data Communication Is Improved Across the Organization
      5. Accessible Data Supports Business Communication Changes
    6. Asynchronous Event-Driven Microservices
      1. Example Team Using Event-Driven Microservices
    7. Synchronous Microservices
      1. Drawbacks of Synchronous Microservices
      2. Benefits of Synchronous Microservices
    8. Summary
  3. 2. Event-Driven Microservice Fundamentals
    1. Building Topologies
      1. Microservice Topology
      2. Business Topology
    2. The Contents of an Event
    3. The Structure of an Event
      1. Unkeyed Event
      2. Entity Event
      3. Keyed Event
    4. Materializing State from Entity Events
    5. Event Data Definitions and Schemas
    6. Microservice Single Writer Principle
    7. Powering Microservices with the Event Broker
      1. Event Storage and Serving
      2. Additional Factors to Consider
    8. Event Brokers Versus Message Brokers
      1. Consuming from the Immutable Log
      2. Providing a Single Source of Truth
    9. Managing Microservices at Scale
      1. Putting Microservices into Containers
      2. Putting Microservices into Virtual Machines
      3. Managing Containers and Virtual Machines
    10. Paying the Microservice Tax
    11. Summary
  4. 3. Communication and Data Contracts
    1. Event-Driven Data Contracts
      1. Using Explicit Schemas as Contracts
      2. Schema Definition Comments
      3. Full-Featured Schema Evolution
      4. Code Generator Support
      5. Breaking Schema Changes
    2. Selecting an Event Format
    3. Designing Events
      1. Tell the Truth, the Whole Truth, and Nothing but the Truth
      2. Use a Singular Event Definition per Stream
      3. Use the Narrowest Data Types
      4. Keep Events Single-Purpose
      5. Minimize the Size of Events
      6. Involve Prospective Consumers in the Event Design
      7. Avoid Events as Semaphores or Signals
    4. Summary
  5. 4. Integrating Event-Driven Architectures with Existing Systems
    1. What Is Data Liberation?
      1. Compromises for Data Liberation
      2. Converting Liberated Data to Events
    2. Data Liberation Patterns
    3. Data Liberation Frameworks
    4. Liberating Data by Query
      1. Bulk Loading
      2. Incremental Timestamp Loading
      3. Autoincrementing ID Loading
      4. Custom Querying
      5. Incremental Updating
      6. Benefits of Query-Based Updating
      7. Drawbacks of Query-Based Updating
    5. Liberating Data Using Change-Data Capture Logs
      1. Benefits of Using Data Store Logs
      2. Drawbacks of Using Data Base Logs
    6. Liberating Data Using Outbox Tables
      1. Performance Considerations
      2. Isolating Internal Data Models
      3. Ensuring Schema Compatibility
      4. Capturing Change-Data Using Triggers
    7. Making Data Definition Changes to Data Sets Under Capture
      1. Handling After-the-Fact Data Definition Changes for the Query and CDC Log Patterns
      2. Handling Data Definition Changes for Change-Data Table Capture Patterns
    8. Sinking Event Data to Data Stores
    9. The Impacts of Sinking and Sourcing on a Business
    10. Summary
  6. 5. Event-Driven Processing Basics
    1. Composing Stateless Topologies
      1. Transformations
      2. Branching and Merging Streams
    2. Repartitioning Event Streams
      1. Example: Repartitioning an Event Stream
    3. Copartitioning Event Streams
      1. Example: Copartitioning an Event Stream
    4. Assigning Partitions to a Consumer Instance
      1. Assigning Partitions with the Partition Assignor
      2. Assigning Copartitioned Partitions
      3. Partition Assignment Strategies
    5. Recovering from Stateless Processing Instance Failures
    6. Summary
  7. 6. Deterministic Stream Processing
    1. Determinism with Event-Driven Workflows
    2. Timestamps
      1. Synchronizing Distributed Timestamps
      2. Processing with Timestamped Events
    3. Event Scheduling and Deterministic Processing
      1. Custom Event Schedulers
      2. Processing Based on Event Time, Processing Time, and Ingestion Time
      3. Timestamp Extraction by the Consumer
      4. Request-Response Calls to External Systems
    4. Watermarks
      1. Watermarks in Parallel Processing
    5. Stream Time
      1. Stream Time in Parallel Processing
    6. Out-of-Order and Late-Arriving Events
      1. Late Events with Watermarks and Stream Time
      2. Causes and Impacts of Out-of-Order Events
      3. Time-Sensitive Functions and Windowing
    7. Handling Late Events
    8. Reprocessing Versus Processing in Near-Real Time
    9. Intermittent Failures and Late Events
    10. Producer/Event Broker Connectivity Issues
    11. Summary and Further Reading
  8. 7. Stateful Streaming
    1. State Stores and Materializing State from an Event Stream
    2. Recording State to a Changelog Event Stream
    3. Materializing State to an Internal State Store
      1. Materializing Global State
      2. Advantages of Using Internal State
      3. Disadvantages of Using Internal State
      4. Scaling and Recovery of Internal State
    4. Materializing State to an External State Store
      1. Advantages of External State
      2. Drawbacks of External State
      3. Scaling and Recovery with External State Stores
    5. Rebuilding Versus Migrating State Stores
      1. Rebuilding
      2. Migrating
    6. Transactions and Effectively Once Processing
      1. Example: Stock Accounting Service
      2. Effectively Once Processing with Client-Broker Transactions
      3. Effectively Once Processing Without Client-Broker Transactions
    7. Summary
  9. 8. Building Workflows with Microservices
    1. The Choreography Pattern
      1. A Simple Event-Driven Choreography Example
      2. Creating and Modifying a Choreographed Workflow
      3. Monitoring a Choreographed Workflow
    2. The Orchestration Pattern
      1. A Simple Event-Driven Orchestration Example
      2. A Simple Direct-Call Orchestration Example
      3. Comparing Direct-Call and Event-Driven Orchestration
      4. Creating and Modifying an Orchestration Workflow
      5. Monitoring the Orchestration Workflow
    3. Distributed Transactions
      1. Choreographed Transactions: The Saga Pattern
      2. Orchestrated Transactions
    4. Compensation Workflows
    5. Summary
  10. 9. Microservices Using Function-as-a-Service
    1. Designing Function-Based Solutions as Microservices
      1. Ensure Strict Membership to a Bounded Context
      2. Commit Offsets Only After Processing Has Completed
      3. Less Is More
    2. Choosing a FaaS Provider
    3. Building Microservices Out of Functions
    4. Cold Start and Warm Starts
    5. Starting Functions with Triggers
      1. Triggering Based on New Events: The Event-Stream Listener
      2. Triggering Based on Consumer Group Lag
      3. Triggering on a Schedule
      4. Triggering Using Webhooks
      5. Triggering on Resource Events
    6. Performing Business Work with Functions
    7. Maintaining State
    8. Functions Calling Other Functions
      1. Event-Driven Communication Pattern
      2. Direct-Call Pattern
    9. Termination and Shutdown
    10. Tuning Your Functions
      1. Allocating Sufficient Resources
      2. Batch Event-Processing Parameters
    11. Scaling Your FaaS Solutions
    12. Summary
  11. 10. Basic Producer and Consumer Microservices
    1. Where Do BPCs Work Well?
      1. Integration with Existing and Legacy Systems
      2. Stateful Business Logic That Isn’t Reliant Upon Event Order
      3. When the Data Layer Does Much of the Work
      4. Independent Scaling of the Processing and Data Layer
    2. Hybrid BPC Applications with External Stream Processing
      1. Example: Using an External Stream-Processing Framework to Join Event Streams
    3. Summary
  12. 11. Heavyweight Framework Microservices
    1. A Brief History of Heavyweight Frameworks
    2. The Inner Workings of Heavyweight Frameworks
    3. Benefits and Limitations
    4. Cluster Setup Options and Execution Modes
      1. Use a Hosted Service
      2. Build Your Own Full Cluster
      3. Create Clusters with CMS Integration
    5. Application Submission Modes
      1. Driver Mode
      2. Cluster Mode
    6. Handling State and Using Checkpoints
    7. Scaling Applications and Handling Event Stream Partitions
      1. Scaling an Application While It Is Running
      2. Scaling an Application by Restarting It
      3. Autoscaling Applications
    8. Recovering from Failures
    9. Multitenancy Considerations
    10. Languages and Syntax
    11. Choosing a Framework
    12. Example: Session Windowing of Clicks and Views
    13. Summary
  13. 12. Lightweight Framework Microservices
    1. Benefits and Limitations
    2. Lightweight Processing
    3. Handling State and Using Changelogs
    4. Scaling Applications and Recovering from Failures
      1. Event Shuffling
      2. State Assignment
      3. State Replication and Hot Replicas
    5. Choosing a Lightweight Framework
      1. Apache Kafka Streams
      2. Apache Samza: Embedded Mode
    6. Languages and Syntax
    7. Stream-Table-Table Join: Enrichment Pattern
    8. Summary
  14. 13. Integrating Event-Driven and Request-Response Microservices
    1. Handling External Events
      1. Autonomously Generated Events
      2. Reactively Generated Events
    2. Handling Autonomously Generated Analytical Events
    3. Integrating with Third-Party Request-Response APIs
    4. Processing and Serving Stateful Data
      1. Serving Real-Time Requests with Internal State Stores
      2. Serving Real-Time Requests with External State Stores
    5. Handling Requests Within an Event-Driven Workflow
      1. Processing Events for User Interfaces
    6. Micro-Frontends in Request-Response Applications
    7. The Benefits of Microfrontends
      1. Composition-Based Microservices
      2. Easy Alignment to Business Requirements
    8. Drawbacks of Microfrontends
      1. Potentially Inconsistent UI Elements and Styling
      2. Varying Microfrontend Performance
      3. Example: Experience Search and Review Application
    9. Summary
  15. 14. Supportive Tooling
    1. Microservice-to-Team Assignment System
    2. Event Stream Creation and Modification
    3. Event Stream Metadata Tagging
    4. Quotas
    5. Schema Registry
    6. Schema Creation and Modification Notifications
    7. Offset Management
    8. Permissions and Access Control Lists for Event Streams
    9. State Management and Application Reset
    10. Consumer Offset Lag Monitoring
    11. Streamlined Microservice Creation Process
    12. Container Management Controls
    13. Cluster Creation and Management
      1. Programmatic Bringup of Event Brokers
      2. Programmatic Bringup of Compute Resources
      3. Cross-Cluster Event Data Replication
      4. Programmatic Bringup of Tooling
    14. Dependency Tracking and Topology Visualization
      1. Topology Example
    15. Summary
  16. 15. Testing Event-Driven Microservices
    1. General Testing Principles
    2. Unit-Testing Topology Functions
      1. Stateless Functions
      2. Stateful Functions
    3. Testing the Topology
    4. Testing Schema Evolution and Compatibility
    5. Integration Testing of Event-Driven Microservices
    6. Local Integration Testing
      1. Create a Temporary Environment Within the Runtime of Your Test Code
      2. Create a Temporary Environment External to Your Test Code
      3. Integrate Hosted Services Using Mocking and Simulator Options
      4. Integrate Remote Services That Have No Local Options
    7. Full Remote Integration Testing
      1. Programmatically Create a Temporary Integration Testing Environment
      2. Testing Using a Shared Environment
      3. Testing Using the Production Environment
    8. Choosing Your Full-Remote Integration Testing Strategy
    9. Summary
  17. 16. Deploying Event-Driven Microservices
    1. Principles of Microservice Deployment
    2. Architectural Components of Microservice Deployment
      1. Continuous Integration, Delivery, and Deployment Systems
      2. Container Management Systems and Commodity Hardware
    3. The Basic Full-Stop Deployment Pattern
    4. The Rolling Update Pattern
    5. The Breaking Schema Change Pattern
      1. Eventual Migration via Two Event Streams
      2. Synchronized Migration to the New Event Stream
    6. The Blue-Green Deployment Pattern
    7. Summary
  18. 17. Conclusion
    1. Communication Layers
    2. Business Domains and Bounded Contexts
    3. Shareable Tools and Infrastructure
    4. Schematized Events
    5. Data Liberation and the Single Source of Truth
    6. Microservices
    7. Microservice Implementation Options
    8. Testing
    9. Deploying
    10. Final Words
  19. Index

Preface
    Conventions Used in This Book
    O’Reilly Online Learning
    How to Contact Us
    Acknowledgments
1. Why Event-Driven Microservices
    What Are Event-Driven Microservices?
    Introduction to Domain-Driven Design and Bounded Contexts
        Leveraging Domain Models and Bounded Contexts
        Aligning Bounded Contexts with Business Requirements
    Communication Structures
        Business Communication Structures
        Implementation Communication Structures
        Data Communication Structures
        Conway’s Law and Communication Structures
    Communication Structures in Traditional Computing
        Option 1: Make a New Service
        Option 2: Add It to the Existing Service
        Pros and Cons of Each Option
        The Team Scenario, Continued
        Conflicting Pressures
    Event-Driven Communication Structures
        Events Are the Basis of Communication
        Event Streams Provide the Single Source of Truth
        Consumers Perform Their Own Modeling and Querying
        Data Communication Is Improved Across the Organization
        Accessible Data Supports Business Communication Changes
    Asynchronous Event-Driven Microservices
        Example Team Using Event-Driven Microservices
    Synchronous Microservices
        Drawbacks of Synchronous Microservices
        Benefits of Synchronous Microservices
    Summary
2. Event-Driven Microservice Fundamentals
    Building Topologies
        Microservice Topology
        Business Topology
    The Contents of an Event
    The Structure of an Event
        Unkeyed Event
        Entity Event
        Keyed Event
    Materializing State from Entity Events
    Event Data Definitions and Schemas
    Microservice Single Writer Principle
    Powering Microservices with the Event Broker
        Event Storage and Serving
        Additional Factors to Consider
    Event Brokers Versus Message Brokers
        Consuming from the Immutable Log
        Providing a Single Source of Truth
    Managing Microservices at Scale
        Putting Microservices into Containers
        Putting Microservices into Virtual Machines
        Managing Containers and Virtual Machines
    Paying the Microservice Tax
    Summary
3. Communication and Data Contracts
    Event-Driven Data Contracts
        Using Explicit Schemas as Contracts
        Schema Definition Comments
        Full-Featured Schema Evolution
        Code Generator Support
        Breaking Schema Changes
    Selecting an Event Format
    Designing Events
        Tell the Truth, the Whole Truth, and Nothing but the Truth
        Use a Singular Event Definition per Stream
        Use the Narrowest Data Types
        Keep Events Single-Purpose
        Minimize the Size of Events
        Involve Prospective Consumers in the Event Design
        Avoid Events as Semaphores or Signals
    Summary
4. Integrating Event-Driven Architectures with Existing Systems
    What Is Data Liberation?
        Compromises for Data Liberation
        Converting Liberated Data to Events
    Data Liberation Patterns
    Data Liberation Frameworks
    Liberating Data by Query
        Bulk Loading
        Incremental Timestamp Loading
        Autoincrementing ID Loading
        Custom Querying
        Incremental Updating
        Benefits of Query-Based Updating
        Drawbacks of Query-Based Updating
    Liberating Data Using Change-Data Capture Logs
        Benefits of Using Data Store Logs
        Drawbacks of Using Data Base Logs
    Liberating Data Using Outbox Tables
        Performance Considerations
        Isolating Internal Data Models
        Ensuring Schema Compatibility
        Capturing Change-Data Using Triggers
    Making Data Definition Changes to Data Sets Under Capture
        Handling After-the-Fact Data Definition Changes for the Query and CDC Log Patterns
        Handling Data Definition Changes for Change-Data Table Capture Patterns
    Sinking Event Data to Data Stores
    The Impacts of Sinking and Sourcing on a Business
    Summary
5. Event-Driven Processing Basics
    Composing Stateless Topologies
        Transformations
        Branching and Merging Streams
    Repartitioning Event Streams
        Example: Repartitioning an Event Stream
    Copartitioning Event Streams
        Example: Copartitioning an Event Stream
    Assigning Partitions to a Consumer Instance
        Assigning Partitions with the Partition Assignor
        Assigning Copartitioned Partitions
        Partition Assignment Strategies
    Recovering from Stateless Processing Instance Failures
    Summary
6. Deterministic Stream Processing
    Determinism with Event-Driven Workflows
    Timestamps
        Synchronizing Distributed Timestamps
        Processing with Timestamped Events
    Event Scheduling and Deterministic Processing
        Custom Event Schedulers
        Processing Based on Event Time, Processing Time, and Ingestion Time
        Timestamp Extraction by the Consumer
        Request-Response Calls to External Systems
    Watermarks
        Watermarks in Parallel Processing
    Stream Time
        Stream Time in Parallel Processing
    Out-of-Order and Late-Arriving Events
        Late Events with Watermarks and Stream Time
        Causes and Impacts of Out-of-Order Events
        Time-Sensitive Functions and Windowing
    Handling Late Events
    Reprocessing Versus Processing in Near-Real Time
    Intermittent Failures and Late Events
    Producer/Event Broker Connectivity Issues
    Summary and Further Reading
7. Stateful Streaming
    State Stores and Materializing State from an Event Stream
    Recording State to a Changelog Event Stream
    Materializing State to an Internal State Store
        Materializing Global State
        Advantages of Using Internal State
        Disadvantages of Using Internal State
        Scaling and Recovery of Internal State
    Materializing State to an External State Store
        Advantages of External State
        Drawbacks of External State
        Scaling and Recovery with External State Stores
    Rebuilding Versus Migrating State Stores
        Rebuilding
        Migrating
    Transactions and Effectively Once Processing
        Example: Stock Accounting Service
        Effectively Once Processing with Client-Broker Transactions
        Effectively Once Processing Without Client-Broker Transactions
    Summary
8. Building Workflows with Microservices
    The Choreography Pattern
        A Simple Event-Driven Choreography Example
        Creating and Modifying a Choreographed Workflow
        Monitoring a Choreographed Workflow
    The Orchestration Pattern
        A Simple Event-Driven Orchestration Example
        A Simple Direct-Call Orchestration Example
        Comparing Direct-Call and Event-Driven Orchestration
        Creating and Modifying an Orchestration Workflow
        Monitoring the Orchestration Workflow
    Distributed Transactions
        Choreographed Transactions: The Saga Pattern
        Orchestrated Transactions
    Compensation Workflows
    Summary
9. Microservices Using Function-as-a-Service
    Designing Function-Based Solutions as Microservices
        Ensure Strict Membership to a Bounded Context
        Commit Offsets Only After Processing Has Completed
        Less Is More
    Choosing a FaaS Provider
    Building Microservices Out of Functions
    Cold Start and Warm Starts
    Starting Functions with Triggers
        Triggering Based on New Events: The Event-Stream Listener
        Triggering Based on Consumer Group Lag
        Triggering on a Schedule
        Triggering Using Webhooks
        Triggering on Resource Events
    Performing Business Work with Functions
    Maintaining State
    Functions Calling Other Functions
        Event-Driven Communication Pattern
        Direct-Call Pattern
    Termination and Shutdown
    Tuning Your Functions
        Allocating Sufficient Resources
        Batch Event-Processing Parameters
    Scaling Your FaaS Solutions
    Summary
10. Basic Producer and Consumer Microservices
    Where Do BPCs Work Well?
        Integration with Existing and Legacy Systems
        Stateful Business Logic That Isn’t Reliant Upon Event Order
        When the Data Layer Does Much of the Work
        Independent Scaling of the Processing and Data Layer
    Hybrid BPC Applications with External Stream Processing
        Example: Using an External Stream-Processing Framework to Join Event Streams
    Summary
11. Heavyweight Framework Microservices
    A Brief History of Heavyweight Frameworks
    The Inner Workings of Heavyweight Frameworks
    Benefits and Limitations
    Cluster Setup Options and Execution Modes
        Use a Hosted Service
        Build Your Own Full Cluster
        Create Clusters with CMS Integration
    Application Submission Modes
        Driver Mode
        Cluster Mode
    Handling State and Using Checkpoints
    Scaling Applications and Handling Event Stream Partitions
        Scaling an Application While It Is Running
        Scaling an Application by Restarting It
        Autoscaling Applications
    Recovering from Failures
    Multitenancy Considerations
    Languages and Syntax
    Choosing a Framework
    Example: Session Windowing of Clicks and Views
    Summary
12. Lightweight Framework Microservices
    Benefits and Limitations
    Lightweight Processing
    Handling State and Using Changelogs
    Scaling Applications and Recovering from Failures
        Event Shuffling
        State Assignment
        State Replication and Hot Replicas
    Choosing a Lightweight Framework
        Apache Kafka Streams
        Apache Samza: Embedded Mode
    Languages and Syntax
    Stream-Table-Table Join: Enrichment Pattern
    Summary
13. Integrating Event-Driven and Request-Response Microservices
    Handling External Events
        Autonomously Generated Events
        Reactively Generated Events
    Handling Autonomously Generated Analytical Events
    Integrating with Third-Party Request-Response APIs
    Processing and Serving Stateful Data
        Serving Real-Time Requests with Internal State Stores
        Serving Real-Time Requests with External State Stores
    Handling Requests Within an Event-Driven Workflow
        Processing Events for User Interfaces
    Micro-Frontends in Request-Response Applications
    The Benefits of Microfrontends
        Composition-Based Microservices
        Easy Alignment to Business Requirements
    Drawbacks of Microfrontends
        Potentially Inconsistent UI Elements and Styling
        Varying Microfrontend Performance
        Example: Experience Search and Review Application
    Summary
14. Supportive Tooling
    Microservice-to-Team Assignment System
    Event Stream Creation and Modification
    Event Stream Metadata Tagging
    Quotas
    Schema Registry
    Schema Creation and Modification Notifications
    Offset Management
    Permissions and Access Control Lists for Event Streams
    State Management and Application Reset
    Consumer Offset Lag Monitoring
    Streamlined Microservice Creation Process
    Container Management Controls
    Cluster Creation and Management
        Programmatic Bringup of Event Brokers
        Programmatic Bringup of Compute Resources
        Cross-Cluster Event Data Replication
        Programmatic Bringup of Tooling
    Dependency Tracking and Topology Visualization
        Topology Example
    Summary
15. Testing Event-Driven Microservices
    General Testing Principles
    Unit-Testing Topology Functions
        Stateless Functions
        Stateful Functions
    Testing the Topology
    Testing Schema Evolution and Compatibility
    Integration Testing of Event-Driven Microservices
    Local Integration Testing
        Create a Temporary Environment Within the Runtime of Your Test Code
        Create a Temporary Environment External to Your Test Code
        Integrate Hosted Services Using Mocking and Simulator Options
        Integrate Remote Services That Have No Local Options
    Full Remote Integration Testing
        Programmatically Create a Temporary Integration Testing Environment
        Testing Using a Shared Environment
        Testing Using the Production Environment
    Choosing Your Full-Remote Integration Testing Strategy
    Summary
16. Deploying Event-Driven Microservices
    Principles of Microservice Deployment
    Architectural Components of Microservice Deployment
        Continuous Integration, Delivery, and Deployment Systems
        Container Management Systems and Commodity Hardware
    The Basic Full-Stop Deployment Pattern
    The Rolling Update Pattern
    The Breaking Schema Change Pattern
        Eventual Migration via Two Event Streams
        Synchronized Migration to the New Event Stream
    The Blue-Green Deployment Pattern
    Summary
17. Conclusion
    Communication Layers
    Business Domains and Bounded Contexts
    Shareable Tools and Infrastructure
    Schematized Events
    Data Liberation and the Single Source of Truth
    Microservices
    Microservice Implementation Options
    Testing
    Deploying
    Final Words
Index

Sources:

Fair Use Sources:

  • B08C9V1FC9 (BEDM)
  • https://learning.oreilly.com/library/view/building-event-driven-microservices/9781492057888
Categories
DevSecOps-Security-Privacy Software Engineering

Cybersecurity Certifications

According to Cyber Risk Analytics‘ “2019 Midyear Quick View Data Breach Report,” the first half of 2019 saw more than 3,800 publicly disclosed breaches with more than 4.1 billion records exposed. This figure represents a 54% increase over reported breaches and a 52% increase in the number of compromised records compared with the same time frame in 2018. More than 60% of the reported breaches were the result of human error, highlighting an ever-increasing need for cybersecurity education, as well as highly skilled and trained cybersecurity professionals.

According to a Cyber Seek report, the number of cybersecurity job openings in the U.S. stands at almost 313,735, with nearly 716,000 cybersecurity professionals employed in today’s workforce. Projections continue to be robust further out: CSO expects that number to hit 500,000 by 2021, with more than 3 million cybersecurity jobs open worldwide that same year.

When evaluating prospective InfoSec candidates, employers frequently look to certification as an important measure of excellence and commitment to quality. We examined five InfoSec certifications we consider to be leaders in the field of information security today:

  1. CEH: Certified Ethical Hacker
  2. CISM: Certified Information Security Manager
  3. CompTIA Security+ and CompTIA PenTest+
  4. CISSP: Certified Information Systems Security Professional
  5. CISA: Certified Information Security Auditor

This year’s list includes entry-level credentials, such as Security+, as well as more advanced certifications, such as the CEH, CISSP, CISM and CISA. We also offer some additional certification options in the last section that cover choices outside our top five, because the field of information security is both wide and varied, with many other options. According to Cyber Seek, more employers are seeking CISA, CISM and CISSP certification holders than there are credential holders which makes these credentials a welcome addition to any certification portfolio.

Absent from our list of the top five is the SANS GIAC Security Essentials (GSEC). The GSEC is still a very worthy credential, but the job board numbers for the CISA were so solid that it merited a spot in the top five.

Security-related job roles cover a lot of ground, such as information security specialist, security analyst, network security administrator, system administrator (with security as a responsibility) and security engineer, as well as specialized roles like malware engineer, intrusion analyst and penetration tester.

Average salaries for information security specialists and security engineers – two of the most common job roles – vary depending on the source. For example, Simply Hired reports $30,263 for specialist positions, whereas Glassdoor’s national average is almost $68,000. For security engineers, Simply Hired reports almost $95,000, while Glassdoor’s average is more than $131,000, with salaries on the high end reported at $144,000.

If you’re serious about advancing your career in the IT field and are interested in specializing in security, certification is a great choice. It’s an effective way to validate your skills and show a current or prospective employer that you’re qualified and properly trained.

Before examining the details of the top five InfoSec certifications, check results from our informal job board survey. It reports the number of job posts nationwide in which our featured certs were mentioned on a given day. This should give you an idea of the relative popularity of each certification.

Job board search results (in alphabetical order, by cybersecurity certification)

CertificationSimply HiredIndeedLinkedIn JobsTechCareersTotal
CEH (EC-Council)2,1722,9915,0931,42211,678
CISA (ISACA)4,5827,13011,4554,13027,297
CISM (ISACA)3,0874,5927,89735015,926
CISSP [(ISC)2]9,67513,54923,5577,68354,464
Security+ (CompTIA)2,4713,2345,34537711,427

Beyond the top 5: More cybersecurity certifications

In addition to these must-have credentials, there are many other certifications available to fit the career needs of any IT professional interested in information security.

While it didn’t make the top five this year, the SANS GIAC Security Essentials (GSEC) remains an excellent entry-level credential for IT professionals seeking to demonstrate that they understand information security terminology and concepts but also possess skills and technical expertise necessary to occupy “hands-on” security roles.

If you find incident response and investigation intriguing, check out the Logical Operations CyberSec First Responder (CFR) certification. This ANSI-accredited and U.S. DoDD-8570 compliant credential recognizes security professionals who can design secure IT environments, perform threat analysis, and respond appropriately and effectively to cyberattacks. Logical Operations offers other certifications, including the Master Mobile Application Developer (MMAD), Certified Virtualization Professional (CVP), Certified Cyber Secure Coder and CloudMASTER.

There are many other certifications to explore or keep your eye on. The associate-level Cisco CCNA Cyber Ops certification is aimed at those who work as analysts in security operations centers (SOCs) in large companies and organizations. Candidates who qualify through the Cisco’s global scholarship program may receive free training, mentoring and testing to help them achieve the CCNA Cyber Ops certification. The CompTIA Cybersecurity Analyst (CySA+), which launched in 2017, is a vendor-neutral certification designed for professionals with three to four years of security and behavioral analytics experience.

The Identity Management Institute (IMI) offers several credentials for identity and access management, data protection, identity protection, identity governance, and more. The IAPP, which focuses on privacy, has a small but growing number of certifications as well.

The SECO-Institute, in cooperation with the Security Academy Netherlands and EXIN, is behind the Cyber Security & Governance Certification Program, an up-and-coming European option that may be headed for the U.S. in the next year or two.

Finally, it may be worth your time to browse the Chartered Institute of Information Security accreditations, which are the U.K. equivalent of the U.S. DoDD 8570 certifications and the corresponding 8140 framework.

Fair Use Sources:

Categories
Bibliography Cloud DevSecOps-Security-Privacy JavaScript Software Engineering

Web Application Security – Exploitation and Countermeasures for Modern Web Applications

Return to Timeline of the History of Computers

Fair Use Source: B085FW7J86

By Andrew Hoffman

While many resources for network and IT security are available, detailed knowledge regarding modern web application security has been lacking—until now. This practical guide provides both offensive and defensive security concepts that software engineers can easily learn and apply.

Andrew Hoffman, a senior security engineer at Salesforce, introduces three pillars of web application security: recon, offense, and defense. You’ll learn methods for effectively researching and analyzing modern web applications—including those you don’t have direct access to. You’ll also learn how to break into web applications using the latest hacking techniques. Finally, you’ll learn how to develop mitigations for use in your own web applications to protect against hackers.

  • Explore common vulnerabilities plaguing today’s web applications
  • Learn essential hacking techniques attackers use to exploit applications
  • Map and document web applications for which you don’t have direct access
  • Develop and deploy customized exploits that can bypass common defenses
  • Develop and deploy mitigations to protect your applications against hackers
  • Integrate secure coding best practices into your development lifecycle
  • Get practical tips to help you improve the overall security of your web applications

From the Preface

Web Application Security walks you through a number of techniques used by talented hackers and bug bounty hunters to break into applications, then teaches you the techniques and processes you can implement in your own software to protect against such hackers.

This book is designed to be read from cover to cover, but can also be used as an on-demand reference for particular types of recon techniques, attacks, and defenses against attacks. Ultimately, this book is written to aid the reader in becoming better at web application security in a way that is practical, hands-on, and follows a logical progression such that no significant prior security experience is required.

Prerequisite Knowledge and Learning Goals

This is a book that will not only aid you in learning how to defend your web application against hackers, but will also walk you through the steps hackers take in order to investigate and break into a web application. Throughout this book we will discuss many techniques that hackers are using today to break into web applications hosted by corporations, governments, and occasionally even hobbyists. Following sufficient investigation into the previously mentioned techniques, we begin a discussion on how to secure web applications against these hackers.

In doing so you will discover brand new ways of thinking about application architecture. You will also learn how to integrate security best practices into an engineering organization. Finally, we will evaluate a number of techniques for defending against the most common and dangerous types of attacks that occur against web applications today.

After completing Web Application Security you will have the required knowledge to perform recon techniques against applications you do not have code-level access to. You will also be able to identify threat vectors and vulnerabilities in web applications, and craft payloads designed to compromise application data, interrupt execution flow, or interfere with the intended function of a web application. With these skills in hand, and the knowledge gained from the final section on securing web applications, you will be able to identify risky areas of a web application’s codebase and understand how to write code to defend against attacks that would otherwise leave your application and its users at risk.

Suggested Background

The potential audience for this book is quite broad, but the style in which the book is written and how the examples are structured should make it ideal for anyone with an intermediary-level background in software engineering.

Minimum Required Skills

In this book, an “intermediary-level background in software engineering” implies the following:

  • You can write basic CRUD (create, read, update, delete) programs in at least one programming language.
  • You can write code that runs on a server somewhere (such as backend code).
  • You can write at least some code that runs in a browser (frontend code, usually JavaScript).
  • You know what HTTP is, and can make, or at least read, GET/POST calls over HTTP in some language or framework.
  • You can write, or at least read and understand, applications that make use of both server-side and client-side code, and communicate between the two over HTTP.
  • You are familiar with at least one popular database (MySql, MongoDB, etc.).

These skills represent the minimum criteria for successfully following the examples in this book. Any experience you have beyond these bullet points is a plus and will make this book that much easier for you to consume and derive educational value from.

About the Author

Andrew Hoffman is a senior product security engineer at Salesforce.com, where he is responsible for the security of multiple JavaScript, NodeJS, and OSS teams. His expertise is in deep DOM and JavaScript security vulnerabilities. He has worked with every major browser vendor, as well as with TC39 and WHATWG ? the organizations responsible for the upcoming version of JavaScript and the browser DOM spec.
Prior to this role, Andrew was a software security engineer working on Locker Service, the world’s first JavaScript namespace isolation library that operates from the interpreter level up. In parallel, Andrew also contributed to the upcoming JavaScript language security feature “Realms,” which provides language level namespace isolation to JavaScript.

Product details:

  • Publication date : March 2, 2020
  • Print length : 331 pages
  • Publisher : O’Reilly Media; 1st edition (March 2, 2020)
  • ASIN : B085FW7J86

Who Benefits Most from Reading This Book?

Prerequisite skills aside, I believe it is important to clarify who will benefit from this book the most, so I’d like to explain who my target audience is. To do so I have structured this section in terms of learning goals and professional interests. If you don’t fit into one of the following categories, you can still learn many valuable or at least interesting concepts from this book.

This book was written to stand the test of time, so if you decide later on to pursue one of the occupations in its target audience, all of the knowledge from this book should still be relevant.

Software Engineers and Web Application Developers

I believe it would be fair to say that the primary audience for this book is an early- to mid-career software engineer or web application developer. Ideally, this reader is interested in gaining a deep understanding of either offensive techniques used by hackers, or defensive techniques used by security engineers to defend against hackers.

Often the titles “web application developer” and “software engineer” are interchangeable, which might lead to a bit of confusion considering I use both of them throughout the upcoming chapters. Let’s start off with some clarification.

Software engineers

In my mind, and for the sake of clarity, when I use the term “software engineer,” I am referring to a generalist who is capable of writing software that runs on a variety of platforms. Software engineers will benefit from this book in several ways.

First off, much of the knowledge contained in this book is transferable with minimal effort to software that does not run on the web. It is also transferable to other types of networked applications, with native mobile applications being the first that come to mind.

Furthermore, several exploits discussed in this book take advantage of server-side integrations involving communication with a web application and another software component. As a result, it is safe to consider any software that interfaces with a web application as a potential threat vector (databases, CRM, accounting, logging tools, etc.).

Web application developers

On the other hand, a “web application developer” by my definition is someone who is highly specialized in writing software that runs on the web. They are often further subdivided into frontend, backend, and full stack developers.

Historically, many attacks against web applications have targeted server-side vulnerabilities. As a result I believe this book’s use case for a backend or full stack developer is very transparent and easily understood.

I also believe this book should be valuable for other types of web application developers, including those who do not write code that runs on a server but instead runs on a web browser (frontend/JavaScript developers).

As I explain in the upcoming chapters, many of the ways in which hackers take advantage of today’s web applications originate via malicious code running in the browser. Some hackers are even taking advantage of the browser DOM or CSS stylesheets in order to attack an application’s users.

These points suggest that it is also important for frontend developers who do not write server-side code to be aware of the security risks their code may expose and how to mitigate those risks.

General Learning Goals

This book should be a fantastic resource for any of the preceding looking to make a career change to a more security-oriented role. It will also be valuable for those looking to learn how to beef up the defenses in their own code or in the code maintained by their organization.

If you want to defend your application against very specific exploits, this book is also for you. This book follows a unique structure, which should enable you to use it as a security reference without ever having to read any of the chapters that involve hacking. That is, of course, if that is your only goal in purchasing this book.

I would suggest reading from cover to cover for the best learning experience, but if you are looking only for a reference on securing against specific types of hacks, just flip the book halfway open and get started reading.

Security Engineers, Pen Testers, and Bug Bounty Hunters

As a result of how this book is structured, it can also be used as a resource for penetration testing, bug bounty hunting, and any other type of application-level security work. If this type of work is relevant or interesting to you, then you may find the first half of the book more to your liking.

This book will take a deep dive into how exploits work from both a code level and an architectural level rather than simply executing well-known open source software (OSS) scripts or making use of paid security automation software. Because of this there is a second audience for this book — software security engineers, IT security engineers, network security engineers, penetration testers, and bug bounty hunters.

Tip

Want to make a little bit of extra money on the side while developing your hacking skills? Read this book and then sign up for one of the bug bounty programs noted in Part III. This is a great way to help other companies improve the security of their products while developing your hacking skills and making some additional cash.

This book will be very beneficial to existing security professionals who understand conceptually how many attacks work but would like a deep dive into the systems and code behind a tool or script.

In today’s security world, it is commonplace for penetration testers to operate using a wide array of prebuilt exploit scripts. This has led to the creation of many paid and open source tools that automate classic attacks, and attacks that can be easily run without deep knowledge regarding the architecture of an application or the logic within a particular block of code.

The exploits and countermeasures contained within this book are presented without the use of any specialized tools. Instead, we will rely on our own scripts, network requests, and the tooling that comes standard in Unix-based operating systems, as well as the standard tooling present in the three major web browsers (Chrome, Firefox, and Edge).

This is not to take away from the value of specialized security tools. In fact, I think that many of them are exceptional and make delivering professional, high-quality penetration tests much easier!

Instead, the reason this book does not contain the use of specialized security tools is so that we can focus on the most important parts of finding a vulnerability, developing an exploit, prioritizing data to compromise, and making sure you can defend against all of the above. As a result, I believe that by the end of this book you will be prepared to go out into the wild and find new types of vulnerabilities, develop exploits against systems that have never been exploited before, and harden the most complex systems against the most persistent attackers.

How Is This Book Organized?

You will soon find that this book is structured quite differently than most other technology books out there. This is intentional. This book is purposefully structured so that there is a nearly 1:1 ratio of chapters regarding hacking (offense) and security (defense).

After beginning our adventure with a bit of a history lesson and some exploration into the technology, tools, and exploits of the past, we will move on to our main topic: exploitation and countermeasures for modern web applications. Hence the subtitle of this book.

The main content in this book is structured into three major parts, with each part containing many individual chapters covering a wide array of topics. Ideally, you will venture through this book in a linear fashion, from page one all the way to the final page. Reading this book in that order will provide the greatest learning possible. As mentioned earlier, this book can also be used as either a hacking reference or a security engineering reference by focusing on the first or second half, respectively.

By now you should understand how to navigate the book, so let’s go over the three main parts of this book so we can grasp the importance of each.

Recon

The first part of this book is “Recon,” where we evaluate ways to gain information regarding a web application without necessarily trying to hack it.

In “Recon,” we discuss a number of important technologies and concepts that are essential to master if you wish to become a hacker. These topics will also be important to anyone looking to lock down an existing application, because the information exposed by many of these techniques can be mitigated with appropriate planning.

I have had the opportunity to work with what I believe to be some of the best penetration testers and bug bounty hunters in the world. Through my conversations with them and my analysis of how they do their work, I’ve come to realize this topic is much more important than many other books make it out to be.

Why is recon important?

I would go so far as to say that for many of the top bug bounty hunters in the world, expert-level reconnaissance ability is what differentiates these “great” hackers from simply “good” hackers.

In other words, it’s one thing to have a fast car (in this case, perhaps knowing how to build exploits), but without knowing the most efficient route to the finish line, you may not win the race. A slower car could make it to the finish line in less time than a fast one if a more efficient path is taken.

If fantasy-based analogies hit closer to home, you could think of recon skills as something akin to a rogue in an RPG. In our case, the rogue’s job isn’t to do lots of damage, but instead to scout ahead of the group and circle back with intel. It’s the guy who helps line up the shots and figures out which battles will have the greatest rewards.

The last part in particular is exceedingly valuable, because it’s likely many types of attacks could be logged against well-defended targets. This means you might only get one chance to exploit a certain software hole before it is found and closed.

We can safely conclude that the second use of reconnaissance is figuring out how to prioritize your exploits.

If you are interested in a career as a penetration tester or a bug bounty hunter, this part of the book will be of utmost importance to you. This is largely because in the world of bug bounty hunting, and to a lesser extent penetration testing, tests are performed “black box” style. “Black box” testing is a style of testing where the tester has no knowledge of the structure and code within an app, and hence must build their own understanding of the application through careful analysis and investigation.

Offense

The second part of this book is “Offense.” Here the focus of the book moves from recon and data gathering to analyzing code and network requests. Then with this knowledge we will attempt to take advantage of insecurely written or improperly configured web applications.

Warning

A number of chapters in this book explain actual hacking techniques used by malicious black hat hackers in the real world. It is imperative that if you are testing techniques found in this book, you do so only against an application that you own or have explicit written permission to test exploits against.

Improper usage of the hacking techniques presented in this book could result in fines, jail time, etc., depending on your country’s laws on hacking activity.

In Part II, we learn how to both build and deploy exploits. These exploits are designed to steal data or forcibly change the behavior of an application.

This part of the book builds on the knowledge from Part I, “Recon.” Using our previously acquired reconnaissance skills in conjunction with newly acquired hacking skills, we will begin taking over and attacking demo web applications.

Part II is organized on an exploit-by-exploit basis. Each chapter explains in detail a different type of exploit.

These chapters start with an explanation of the exploit itself so you can understand how it works mechanically. Then we discuss how to search for vulnerabilities where this exploit can be applied. Finally, we craft a payload specific to the demo application we are exploiting. We then deploy the payload, and observe the results.

Vulnerabilities considered in depth

Cross-Site Scripting (XSS), one of the first exploits we dig into, is a type of attack that works against a wide array of web applications, but can be applied to other applications as well (e.g., mobile apps, flash/ActionScript games, etc.). This particular attack involves writing some malicious code on your own machine, then taking advantage of poor filtration mechanisms in an app that will allow your script to execute on another user’s machine.

When we discuss an exploit like an XSS attack, we will start with a vulnerable app. This demo app will be straightforward and to the point, ideally just a few paragraphs of code. From this foundation, we will write a block of code to be injected as a payload into the demo app, which will then take advantage of a hypothetical user on the other side.

Sounds simple doesn’t it? And it should be. Without any defenses, most software systems are easy to break into. As a result, with an exploit like XSS where there are many defenses, we will progressively dig deeper and deeper into the specifics of writing and deploying an attack.

We will initially attempt to break down routine defenses and eventually move on to bypassing more advanced defense mechanisms. Remember, just because someone built a wall to defend their codebase doesn’t mean you can’t go over it or underneath it. This is where we will get to use some creativity and find some unique and interesting solutions.

Part II is important because understanding the mindset of a hacker is often vital for architecting secure codebases. It is exceptionally important for any reader interested in hacking, penetration testing, or bug bounty hunting.

Defense

The third and final part of this book, “Defense,” is about securing your own code against hackers. In Part III, we go back and look at every type of exploit we covered in Part II and attempt to consider them again with a completely opposite viewpoint. This time, we will not be concentrating on breaking into software systems, but instead attempting to prevent or mitigate the probability that a hacker could break into our systems.

In Part III you will learn how to protect against specific exploits from Part II, in addition to learning general protections that will secure your codebase against a wide variety of attacks. These general protections range from “secure by default” engineering methodologies, to secure coding best practices that can be enforced easily by an engineering team using tests and other simple automated tooling (such as a linter).

Beyond learning how to write more secure code, you will also learn a number of increasingly valuable tricks for catching hackers in the act and improving your organization’s attitude toward software security.

Most chapters in Part III restructured somewhat akin to the hacking chapters in Part II. We begin with an overview of the technology and skills required as we begin preparing a defense against a specific type of attack.

Initially we will prepare a basic-level defense, which should help mitigate attacks but may not always fend off the most persistent hackers. Finally, we will improve our defenses to the point where most, if not all, hacking attempts will be stopped.

At this point, the structure of Part III begins to differ from that of Part II as we discuss trade-offs that result from improving application security. Generally speaking, all measures of improving security will have some type of trade-off outside of security. It may not be your place to make suggestions on what level of risk should be accepted at the cost of your product, but you should be aware of the trade-offs being made.

Often, these trade-offs come in the form of application performance. The more efforts you take to read and sanitize data, the more operations are performed outside of the standard functionality of your application. Hence a secure feature typically requires more computing resources than an insecure feature.

With further operations also comes more code, which means more maintenance, tests, and engineering time. This development overhead to security often comes in the form of logging or monitoring overhead as well.

Finally, some security precautions will come at the cost of reduced usability.

Trade-off evaluation

A very simple example of this process of comparing security benefits to their cost, in terms of usability and performance, is a login form. If an error message for an invalid username is displayed to the user when attempting to log in, it becomes significantly easier for a hacker to brute force username/password combinations. This occurs because the hacker no longer has to find a list of active login usernames, as the application will confirm a user account. The hacker simply needs to successfully brute force a few usernames, which can be confirmed and logged for later break-in attempts.

Next, the hacker only needs to brute force passwords rather than username/password combinations, which implies significantly decreased mathematical complexity and takes much less time and resources.

Furthermore, if the application uses an email and password scheme for login rather than a username and password scheme, then we have another problem. A hacker can use this login form to find valid email addresses that can be sold for marketing or spam purposes. Even if precautions are taken to prevent brute forcing, carefully crafted inputs (e.g., first.last@company.com, firstlast@company.com, firstl@company.com) can allow the hacker to reverse engineer the schema used for company email accounts and pinpoint the valid accounts of execs for sales or individuals with important access criteria for phishing.

As a result, it is often considered best practice to provide more generic error messages to the user. Of course, this change conflicts with the user experience because more specific error messages are definitely ideal for the usability of your application.

This is a great example of a trade-off that can be made for improved application security, but at the cost of reduced usability. This should give you an idea of the type of trade-offs that are discussed in Part III of this book.

This part of the book is extremely important for any security engineer who wants to beef up their skills, or any software engineer looking at transitioning to a security engineering role. The information presented here will help in architecting and writing more secure applications.

As in Part II, understanding how an application’s security can be improved is a valuable asset for any type of hacker. This is because while routine defenses can often be easily bypassed, more complex defenses require deeper understanding and knowledge to bypass. This is further evidence as to why I suggest reading the book from start to finish.

Although some parts of this book may give you more valuable learning than others, depending on your goals, I doubt any of it will be wasted. Cross-training of this sort is particularly valuable, as each part of the book is just another perspective on the same puzzle.

Categories
Cloud Data Science - Big Data DevSecOps-Security-Privacy History Software Engineering

Data Breaches – 2014 AD

Return to Timeline of the History of Computers

2014

Data Breaches

“In 2014, data breaches touched individuals on a scale not seen before, in terms of both the amount and the sensitivity of the data that was stolen. These hacks served as a wake-up call to the world about the reality of living a digitally dependent way of life—both for individuals and for corporate data masters.”

“Most news coverage of data breaches focused on losses suffered by corporations and government agencies in North America—not because these systems were especially vulnerable, but because laws required public disclosure. High-profile attacks affected millions of accounts with companies including Target (in late 2013), JPMorgan Chase, and eBay. Midway through the year”, it was revealed that the Obama Administration’s United States Office of Personnel Management (OPM) was hacked via out-sourced contractors connected to the Chinese Communist government and “that highly personal (and sensitive) information belonging to 18 million former, current, and prospective federal and military employees had been stolen. Meanwhile, information associated with at least half a billion user accounts at Yahoo! was being hacked, although this information wouldn’t come out until 2016.”

Data from organizations outside the US was no less immune. The European Central Bank, HSBC Turkey, and others were hit. These hacks represented millions of victims across a spectrum of industries, such as banking, government, entertainment, retail, and health. While some of the industry and government datasets ended up online, available to the highest bidder in the criminal underground, many other datasets did not, fueling speculation and public discourse about why and what could be done with such data.

The 2014 breaches also expanded the public’s understanding about the value of certain types of hacked data beyond the traditional categories of credit card numbers, names, and addresses. The November 24, 2014, hack of Sony Pictures, for example, didn’t just temporarily shut down the film studio: the hackers also exposed personal email exchanges, harmed creative intellectual property, and rekindled threats against the studio’s freedom of expression, allegedly in retaliation for the studio’s decision to participate in the release of a Hollywood movie critical of a foreign government.

Perhaps most importantly, the 2014 breaches exposed the generally poor state of software security, best practices, and experts’ digital acumen across the world. The seams between the old world and that of a world with modern, networked technology were not as neatly stitched as many had assumed.”

SEE ALSO Morris Worm (1988), Cyber Weapons (2010)

Since 2014, high-profile data breaches have affected billions of people worldwide.

Fair Use Sources: B07C2NQSPV