Categories
Bibliography Python Software Engineering

Introducing Python: Modern Computing in Simple Packages, by Bill Lubanovic

See also: Python, Python Bibliography and Bibliography of Python Libraries and Web Frameworks

See: Introducing Python: Modern Computing in Simple Packages, by Bill Lubanovic, 2020, B0815R5543 (IPyBLub)

Fair Use Source: B0815R5543 (IPyBLub)

About This Book:

Easy to understand and fun to read, this updated edition of Introducing Python is ideal for beginning programmers as well as those new to the language. Author Bill Lubanovic takes you from the basics to more involved and varied topics, mixing tutorials with cookbook-style code recipes to explain concepts in Python 3. End-of-chapter exercises help you practice what you’ve learned.

You’ll gain a strong foundation in the language, including best practices for testing, debugging, code reuse, and other development tips. This book also shows you how to use Python for applications in business, science, and the arts, using various Python tools and open source packages.

About the Author:

Bill Lubanovic has developed software with UNIX since 1977, GUIs since 1981, databases since 1990, and the Web since 1993. Recently, he developed core services and distributed systems with a remote team for a Manhattan startup. Currently, he’s integrating OpenStack services for a supercomputer company.

Book Details:

  • ASIN: B0815R5543
  • ISBN-10: 1492051365
  • ISBN-13: 978-1492051367
  • Publisher: O’Reilly Media; 2nd edition (November 6, 2019)
  • Publication date: November 6, 2019
  • Print length: 960 pages

Table of Contents:

Preface Audience

Changes in the Second Edition

Outline

Python Versions

Conventions Used in This Book

Using Code Examples

O’Reilly Online Learning

How to Contact Us

Acknowledgments

I. Python Basics

  1. A Taste of Py Mysteries

Little Programs

A Bigger Program

Python in the Real World

Python Versus the Language from Planet X

Why Python?

Why Not Python?

Python 2 Versus Python 3

Installing Python

Running Python Using the Interactive Interpreter

Using Python Files

What’s Next?

Your Moment of Zen

Coming Up

Things to Do

  1. Data: Types, Values, Variables, and Names Python Data Are Objects

Types

Mutability

Literal Values

Variables

Assignment

Variables Are Names, Not Places

Assigning to Multiple Names

Reassigning a Name

Copying

Choose Good Variable Names

Coming Up

Things to Do

  1. Numbers Booleans

Integers Literal Integers

Integer Operations

Integers and Variables

Precedence

Bases

Type Conversions

How Big Is an int?

Floats

Math Functions

Coming Up

Things to Do

  1. Choose with if Comment with #

Continue Lines with \

Compare with if, elif, and else

What Is True?

Do Multiple Comparisons with in

New: I Am the Walrus

Coming Up

Things to Do

  1. Text Strings Create with Quotes

Create with str()

Escape with \

Combine by Using +

Duplicate with *

Get a Character with []

Get a Substring with a Slice

Get Length with len()

Split with split()

Combine by Using join()

Substitute by Using replace()

Strip with strip()

Search and Select

Case

Alignment

Formatting Old style: %

New style: {} and format()

Newest Style: f-strings

More String Things

Coming Up

Things to Do

  1. Loop with while and for Repeat with while Cancel with break

Skip Ahead with continue

Check break Use with else

Iterate with for and in Cancel with break

Skip with continue

Check break Use with else

Generate Number Sequences with range()

Other Iterators

Coming Up

Things to Do

  1. Tuples and Lists Tuples Create with Commas and ()

Create with tuple()

Combine Tuples by Using +

Duplicate Items with *

Compare Tuples

Iterate with for and in

Modify a Tuple

Lists Create with []

Create or Convert with list()

Create from a String with split()

Get an Item by [ offset ]

Get Items with a Slice

Add an Item to the End with append()

Add an Item by Offset with insert()

Duplicate All Items with *

Combine Lists by Using extend() or +

Change an Item by [ offset ]

Change Items with a Slice

Delete an Item by Offset with del

Delete an Item by Value with remove()

Get an Item by Offset and Delete It with pop()

Delete All Items with clear()

Find an Item’s Offset by Value with index()

Test for a Value with in

Count Occurrences of a Value with count()

Convert a List to a String with join()

Reorder Items with sort() or sorted()

Get Length with len()

Assign with =

Copy with copy(), list(), or a Slice

Copy Everything with deepcopy()

Compare Lists

Iterate with for and in

Iterate Multiple Sequences with zip()

Create a List with a Comprehension

Lists of Lists

Tuples Versus Lists

There Are No Tuple Comprehensions

Coming Up

Things to Do

  1. Dictionaries and Sets Dictionaries Create with {}

Create with dict()

Convert with dict()

Add or Change an Item by [ key ]

Get an Item by [key] or with get()

Get All Keys with keys()

Get All Values with values()

Get All Key-Value Pairs with items()

Get Length with len()

Combine Dictionaries with {**a, **b}

Combine Dictionaries with update()

Delete an Item by Key with del

Get an Item by Key and Delete It with pop()

Delete All Items with clear()

Test for a Key with in

Assign with =

Copy with copy()

Copy Everything with deepcopy()

Compare Dictionaries

Iterate with for and in

Dictionary Comprehensions

Sets Create with set()

Convert with set()

Get Length with len()

Add an Item with add()

Delete an Item with remove()

Iterate with for and in

Test for a Value with in

Combinations and Operators

Set Comprehensions

Create an Immutable Set with frozenset()

Data Structures So Far

Make Bigger Data Structures

Coming Up

Things to Do

  1. Functions Define a Function with def

Call a Function with Parentheses

Arguments and Parameters None Is Useful

Positional Arguments

Keyword Arguments

Specify Default Parameter Values

Explode/Gather Positional Arguments with *

Explode/Gather Keyword Arguments with **

Keyword-Only Arguments

Mutable and Immutable Arguments

Docstrings

Functions Are First-Class Citizens

Inner Functions Closures

Anonymous Functions: lambda

Generators Generator Functions

Generator Comprehensions

Decorators

Namespaces and Scope

Uses of _ and __ in Names

Recursion

Async Functions

Exceptions Handle Errors with try and except

Make Your Own Exceptions

Coming Up

Things to Do

  1. Oh Oh: Objects and Classes What Are Objects?

Simple Objects Define a Class with class

Attributes

Methods

Initialization

Inheritance Inherit from a Parent Class

Override a Method

Add a Method

Get Help from Your Parent with super()

Multiple Inheritance

Mixins

In self Defense

Attribute Access Direct Access

Getters and Setters

Properties for Attribute Access

Properties for Computed Values

Name Mangling for Privacy

Class and Object Attributes

Method Types Instance Methods

Class Methods

Static Methods

Duck Typing

Magic Methods

Aggregation and Composition

When to Use Objects or Something Else

Named Tuples

Dataclasses

Attrs

Coming Up

Things to Do

  1. Modules, Packages, and Goodies Modules and the import Statement Import a Module

Import a Module with Another Name

Import Only What You Want from a Module

Packages The Module Search Path

Relative and Absolute Imports

Namespace Packages

Modules Versus Objects

Goodies in the Python Standard Library Handle Missing Keys with setdefault() and defaultdict()

Count Items with Counter()

Order by Key with OrderedDict()

Stack + Queue == deque

Iterate over Code Structures with itertools

Print Nicely with pprint()

Get Random

More Batteries: Get Other Python Code

Coming Up

Things to Do

II. Python in Practice

  1. Wrangle and Mangle Data Text Strings: Unicode Python 3 Unicode Strings

UTF-8

Encode

Decode

HTML Entities

Normalization

For More Information

Text Strings: Regular Expressions Find Exact Beginning Match with match()

Find First Match with search()

Find All Matches with findall()

Split at Matches with split()

Replace at Matches with sub()

Patterns: Special Characters

Patterns: Using Specifiers

Patterns: Specifying match() Output

Binary Data bytes and bytearray

Convert Binary Data with struct

Other Binary Data Tools

Convert Bytes/Strings with binascii()

Bit Operators

A Jewelry Analogy

Coming Up

Things to Do

  1. Calendars and Clocks Leap Year

The datetime Module

Using the time Module

Read and Write Dates and Times

All the Conversions

Alternative Modules

Coming Up

Things to Do

  1. Files and Directories File Input and Output Create or Open with open()

Write a Text File with print()

Write a Text File with write()

Read a Text File with read(), readline(), or readlines()

Write a Binary File with write()

Read a Binary File with read()

Close Files Automatically by Using with

Change Position with seek()

Memory Mapping

File Operations Check Existence with exists()

Check Type with isfile()

Copy with copy()

Change Name with rename()

Link with link() or symlink()

Change Permissions with chmod()

Change Ownership with chown()

Delete a File with remove()

Directory Operations Create with mkdir()

Delete with rmdir()

List Contents with listdir()

Change Current Directory with chdir()

List Matching Files with glob()

Pathnames Get a Pathname with abspath()

Get a symlink Pathname with realpath()

Build a Pathname with os.path.join()

Use pathlib

BytesIO and StringIO

Coming Up

Things to Do

  1. Data in Time: Processes and Concurrency Programs and Processes Create a Process with subprocess

Create a Process with multiprocessing

Kill a Process with terminate()

Get System Info with os

Get Process Info with psutil

Command Automation Invoke

Other Command Helpers

Concurrency Queues

Processes

Threads

concurrent.futures

Green Threads and gevent

twisted

asyncio

Redis

Beyond Queues

Coming Up

Things to Do

  1. Data in a Box: Persistent Storage Flat Text Files

Padded Text Files

Tabular Text Files CSV

XML

An XML Security Note

HTML

JSON

YAML

Tablib

Pandas

Configuration Files

Binary Files Padded Binary Files and Memory Mapping

Spreadsheets

HDF5

TileDB

Relational Databases SQL

DB-API

SQLite

MySQL

PostgreSQL

SQLAlchemy

Other Database Access Packages

NoSQL Data Stores The dbm Family

Memcached

Redis

Document Databases

Time Series Databases

Graph Databases

Other NoSQL

Full-Text Databases

Coming Up

Things to Do

  1. Data in Space: Networks TCP/IP Sockets

Scapy

Netcat

Networking Patterns

The Request-Reply Pattern ZeroMQ

Other Messaging Tools

The Publish-Subscribe Pattern Redis

ZeroMQ

Other Pub-Sub Tools

Internet Services Domain Name System

Python Email Modules

Other Protocols

Web Services and APIs

Data Serialization Serialize with pickle

Other Serialization Formats

Remote Procedure Calls XML RPC

JSON RPC

MessagePack RPC

Zerorpc

gRPC

Twirp

Remote Management Tools

Big Fat Data Hadoop

Spark

Disco

Dask

Clouds Amazon Web Services

Google Cloud

Microsoft Azure

OpenStack

Docker Kubernetes

Coming Up

Things to Do

  1. The Web, Untangled Web Clients Test with telnet

Test with curl

Test with httpie

Test with httpbin

Python’s Standard Web Libraries

Beyond the Standard Library: requests

Web Servers The Simplest Python Web Server

Web Server Gateway Interface (WSGI)

ASGI

Apache

NGINX

Other Python Web Servers

Web Server Frameworks Bottle

Flask

Django

Other Frameworks

Database Frameworks

Web Services and Automation webbrowser

webview

Web APIs and REST

Crawl and Scrape Scrapy

BeautifulSoup

Requests-HTML

Let’s Watch a Movie

Coming Up

Things to Do

  1. Be a Pythonista About Programming

Find Python Code

Install Packages Use pip

Use virtualenv

Use pipenv

Use a Package Manager

Install from Source

Integrated Development Environments IDLE

PyCharm

IPython

Jupyter Notebook

JupyterLab

Name and Document

Add Type Hints

Test Check with pylint, pyflakes, flake8, or pep8

Test with unittest

Test with doctest

Test with nose

Other Test Frameworks

Continuous Integration

Debug Python Code Use print()

Use Decorators

Use pdb

Use breakpoint()

Log Error Messages

Optimize Measure Timing

Algorithms and Data Structures

Cython, NumPy, and C Extensions

PyPy

Numba

Source Control Mercurial

Git

Distribute Your Programs

Clone This Book

How You Can Learn More Books

Websites

Groups

Conferences

Getting a Python Job

Coming Up

Things to Do

  1. Py Art 2-D Graphics Standard Library

PIL and Pillow

ImageMagick

3-D Graphics

3-D Animation

Graphical User Interfaces

Plots, Graphs, and Visualization Matplotlib

Seaborn

Bokeh

Games

Audio and Music

Coming Up

Things to Do

  1. Py at Work The Microsoft Office Suite

Carrying Out Business Tasks

Processing Business Data Extracting, Transforming, and Loading

Data Validation

Additional Sources of Information

Open Source Python Business Packages

Python in Finance

Business Data Security

Maps Formats

Draw a Map from a Shapefile

Geopandas

Other Mapping Packages

Applications and Data

Coming Up

Things to Do

  1. Py Sci Math and Statistics in the Standard Library Math Functions

Working with Complex Numbers

Calculate Accurate Floating Point with decimal

Perform Rational Arithmetic with fractions

Use Packed Sequences with array

Handling Simple Stats with statistics

Matrix Multiplication

Scientific Python

NumPy Make an Array with array()

Make an Array with arange()

Make an Array with zeros(), ones(), or random()

Change an Array’s Shape with reshape()

Get an Element with []

Array Math

Linear Algebra

SciPy

SciKit

Pandas

Python and Scientific Areas

Coming Up

Things to Do

A. Hardware and Software for Beginning Programmers Hardware Caveman Computers

Electricity

Inventions

An Idealized Computer

The CPU

Memory and Caches

Storage

Inputs

Outputs

Relative Access Times

Software In the Beginning Was the Bit

Machine Language

Assembler

Higher-Level Languages

Operating Systems

Virtual Machines

Containers

Distributed Computing and Networks

The Cloud

Kubernetes

B. Install Python 3 Check Your Python Version

Install Standard Python macOS

Windows

Linux or Unix

Install the pip Package Manager

Install virtualenv

Other Packaging Solutions

Install Anaconda Install Anaconda’s Package Manager conda

C. Something Completely Different: Async Coroutines and Event Loops

Asyncio Alternatives

Async Versus…

Async Frameworks and Servers

D. Answers to Exercises 1. A Taste of Py

  1. Data: Types, Values, Variables, and Names
  2. Numbers
  3. Choose with if
  4. Text Strings
  5. Loop with while and for
  6. Tuples and Lists
  7. Dictionaries
  8. Functions
  9. Oh Oh: Objects and Classes
  10. Modules, Packages, and Goodies
  11. Wrangle and Mangle Data
  12. Calendars and Clocks
  13. Files and Directories
  14. Data in Time: Processes and Concurrency
  15. Data in a Box: Persistent Storage
  16. Data in Space: Networks
  17. The Web, Untangled
  18. Be a Pythonista
  19. Py Art
  20. Py at Work
  21. PySci

E. Cheat Sheets Operator Precedence

String Methods Change Case

Search

Modify

Format

String Type

String Module Attributes

Coda

Index

Preface:

“As the title promises, this book will introduce you to one of the world’s most popular programming languages: Python. It’s aimed at beginning programmers as well as more experienced programmers who want to add Python to the languages they already know.” (IPyBLub)

“In most cases, it’s easier to learn a computer language than a human language. There’s less ambiguity and fewer exceptions to keep in your head. Python is one of the most consistent and clear computer languages. It balances ease of learning, ease of use, and expressive power.” (IPyBLub)

“Computer languages are made of data (like nouns in spoken languages) and instructions or code (like verbs). You need both. In alternating chapters, you’ll be introduced to Python’s basic code and data structures, learn how to combine them, and build up to more advanced ones. The programs that you read and write will get longer and more complex. Using a woodworking analogy, we’ll start with a hammer, nails, and scraps of wood. Over the first half of this book, we’ll introduce more specialized components, up to the equivalents of lathes and other power tools.” (IPyBLub)

“You’ll not only learn the language, but also what to do with it. We’ll begin with the Python language and its “batteries included” standard library, but I’ll also show you how to find, download, install, and use some good third-party packages. My emphasis is on whatever I’ve actually found useful in more than 10 years of production Python development, rather than fringe topics or complex hacks.” (IPyBLub)

“Although this is an introduction, some advanced topics are included because I want to expose them to you. Areas like databases and the web are still covered, but technology changes fast. A Python programmer might now be expected to know something about cloud computing, machine learning, or event streaming. You’ll find something here on all of these.” (IPyBLub)

“Python has some special features that work better than adapting styles from other languages that you may know. For example, using for and iterators is a more direct way of making a loop than manually incrementing some counter variable.” (IPyBLub)

“When you’re learning something new, it’s hard to tell which terms are specific instead of colloquial, and which concepts are actually important. In other words, “Is this on the test?” I’ll highlight terms and ideas that have specific meaning or importance in Python, but not too many at once. Real Python code is included early and often.” (IPyBLub)

Note: “I’ll include a note such as this when something might be confusing, or if there’s a more appropriate Pythonic way to do it.” (IPyBLub)

“Python isn’t perfect. I’ll show you things that seem odd or that should be avoided — and offer alternatives you can use, instead.” (IPyBLub)

“Now and then, my opinions on some subjects (such as object inheritance, or MVC and REST designs for the web) may vary a bit from the common wisdom. See what you think.” (IPyBLub)

Audience:

“This book is for anybody interested in learning one of the world’s most popular computing languages, regardless of whether you have previously learned any programming.” (IPyBLub)

Changes in the Second Edition:

“What’s changed since the first edition?

About a hundred more pages, including cat pictures.

Twice the chapters, each shorter now.

An early chapter devoted to data types, variables, and names.

New standard Python features like f-strings.

New or improved third-party packages.

New code examples throughout.

An appendix on basic hardware and software, for new programmers.

An appendix on asyncio, for not-so-new programmers.

“New stack” coverage: containers, clouds, data science, and machine learning.

Hints on getting a job programming in Python.

What hasn’t changed? Examples using bad poetry and ducks. These are evergreen.” (IPyBLub)

Outline:

“Part I (Chapters 1–11) explains Python’s basics. You should read these chapters in order. I work up from the simplest data and code structures, combining them on the way into more detailed and realistic programs. Part II (Chapters 12–22) shows how Python is used in specific application areas such as the web, databases, networks, and so on; read these chapters in any order you like.” (IPyBLub)

“Here’s a brief preview of the chapters and appendixes, including some of the terms that you’ll run into there:” (IPyBLub)

Chapter 1, A Taste of Py

“Computer programs are not that different from directions that you see every day. Some little Python programs give you a glimpse of the language’s looks, capabilities, and uses in the real world. You’ll see how to run a Python program within its interactive interpreter (or shell), or from a text file saved on your computer.” (IPyBLub)

Chapter 2, Data: Types, Values, Variables, and Names

“Computer languages mix data and instructions. Different types of data are stored and treated differently by the computer. They may allow their values to be changed (mutable) or not (immutable). In a Python program, data can be literal (numbers like 78, text strings like “waffle”) or represented by named variables. Python treats variables like names, which is different from many other languages and has some important consequences.” (IPyBLub)

Chapter 3, Numbers

“This chapter shows Python’s simplest data types: booleans, integers, and floating-point numbers. You’ll also learn the basic math operations. The examples use Python’s interactive interpreter like a calculator.” (IPyBLub)

Chapter 4, Choose with if

“We’ll bounce between Python’s nouns (data types) and verbs (program structures) for a few chapters. Python code normally runs a line at a time, from the start to the end of a program. The if code structure lets you run different lines of code, depending on some data comparison.” (IPyBLub)

Chapter 5, Text Strings

“Back to nouns, and the world of text strings. Learn how to create, combine, change, retrieve, and print strings.” (IPyBLub)

Chapter 6, Loop with while and for

“Verbs again, and two ways to make a loop: for and while. You’ll be introduced to a core Python concept: iterators.” (IPyBLub)

Chapter 7, Tuples and Lists

“It’s time for the first of Python’s higher-level built-in data structures: lists and tuples. These are sequences of values, like LEGO for building much more complex data structures. Step through them with iterators, and build lists quickly with comprehensions.” (IPyBLub)

Chapter 8, Dictionaries and Sets

“Dictionaries (aka dicts) and sets let you save data by their values rather than their position. This turns out to be very handy and will be among your favorite Python features.” (IPyBLub)

Chapter 9, Functions

“Weave the data and code structures of the previous chapters to compare, choose, or repeat. Package code in functions and handle errors with exceptions.” (IPyBLub)

Chapter 10, Oh Oh: Objects and Classes

“The word object is a bit fuzzy, but important in many computer languages, including Python. If you’ve done object-oriented programming in other languages, Python is a bit more relaxed. This chapter explains how to use objects and classes, and when it’s better to use alternatives.” (IPyBLub)

Chapter 11, Modules, Packages, and Goodies

“This chapter demonstrates how to scale out to larger code structures: modules, packages, and programs. You’ll see where to put code and data, how to get data in and out, handle options, tour the Python Standard Library, and take a glance at what lies beyond.” (IPyBLub)

Chapter 12, Wrangle and Mangle Data

“Learn to manage (or mangle) data like a pro. This chapter is all about text and binary data, joy with Unicode characters, and regex text searching. It also introduces the data types bytes and bytearray, counterparts of strings that contain raw binary values instead of text characters.” (IPyBLub)

Chapter 13, Calendars and Clocks

“Dates and times can be messy to handle. This chapter shows common problems and useful solutions.” (IPyBLub)

Chapter 14, Files and Directories

“Basic data storage uses files and directories. This chapter shows you how to create and use them.” (IPyBLub)

Chapter 15, Data in Time: Processes and Concurrency

“This is the first hard-core system chapter. Its theme is data in time — how to use programs, processes, and threads to do more things at a time (concurrency). Python’s recent async additions are mentioned, with details in Appendix C.” (IPyBLub)

Chapter 16, Data in a Box: Persistent Storage

“Data can be stored and retrieved with basic flat files and directories within filesystems. They gain some structure with common text formats such as CSV, JSON, and XML. As data get larger and more complex, they need the services of databases — traditional relational ones, and some newer NoSQL data stores.” (IPyBLub)

Chapter 17, Data in Space: Networks

“Send your code and data through space in networks with services, protocols, and APIs. Examples range from low-level TCP sockets, to messaging libraries and queuing systems, to cloud deployment.” (IPyBLub)

Chapter 18, The Web, Untangled

“The web gets its own chapter — clients, servers, APIs, and frameworks. You’ll crawl and scrape websites, and then build real websites with request parameters and templates.” (IPyBLub)

Chapter 19, Be a Pythonista

“This chapter contains tips for Python developers — installation with pip and virtualenv, using IDEs, testing, debugging, logging, source control, and documentation. It also helps you to find and install useful third-party packages, package your own code for reuse, and learn where to get more information.” (IPyBLub)

Chapter 20, Py Art

“People are doing cool things with Python in the arts: graphics, music, animation, and games.” (IPyBLub)

Chapter 21, Py at Work

“Python has specific applications for business: data visualization (plots, graphs, and maps), security, and regulation.” (IPyBLub)

Chapter 22, Py Sci

“In the past few years, Python has emerged as a top language for science: math and statistics, physical science, bioscience, and medicine. Data science and machine learning are notable strengths. This chapter touches on NumPy, SciPy, and Pandas.” (IPyBLub)

Appendix A, Hardware and Software for Beginning Programmers

“If you’re fairly new to programming, this describes how hardware and software actually work. It introduces some terms that you’ll keep running into.” (IPyBLub)

Appendix B, Install Python 3

“If you don’t already have Python 3 on your computer, this appendix shows you how to install it, whether you’re running Windows, macOS, Linux, or some other variant of Unix.” (IPyBLub)

Appendix C, Something Completely Different: Async

“Python has been adding asynchronous features in different releases, and they’re not easy to understand. I mention them as they come up in various chapters, but save a detailed discussion for this appendix.” (IPyBLub)

Appendix D, Answers to Exercises

“This has the answers to the end-of-chapter exercises. Don’t peek here until you’ve tried the exercises yourself, or you might be turned into a newt.” (IPyBLub)

Appendix E, Cheat Sheets

“This appendix contains cheat sheets to use as a quick reference.” (IPyBLub)

Sources:

Fair Use Sources: