See also: Python, Python Bibliography and Bibliography of Python Libraries and Web Frameworks
See: Introducing Python: Modern Computing in Simple Packages, by Bill Lubanovic, 2020, B0815R5543 (IPyBLub)
Fair Use Source: B0815R5543 (IPyBLub)
About This Book:
Easy to understand and fun to read, this updated edition of Introducing Python is ideal for beginning programmers as well as those new to the language. Author Bill Lubanovic takes you from the basics to more involved and varied topics, mixing tutorials with cookbook-style code recipes to explain concepts in Python 3. End-of-chapter exercises help you practice what you’ve learned.
You’ll gain a strong foundation in the language, including best practices for testing, debugging, code reuse, and other development tips. This book also shows you how to use Python for applications in business, science, and the arts, using various Python tools and open source packages.
About the Author:
Bill Lubanovic has developed software with UNIX since 1977, GUIs since 1981, databases since 1990, and the Web since 1993. Recently, he developed core services and distributed systems with a remote team for a Manhattan startup. Currently, he’s integrating OpenStack services for a supercomputer company.
Book Details:
- ASIN: B0815R5543
- ISBN-10: 1492051365
- ISBN-13: 978-1492051367
- Publisher: O’Reilly Media; 2nd edition (November 6, 2019)
- Publication date: November 6, 2019
- Print length: 960 pages
Table of Contents:
Preface Audience
Changes in the Second Edition
Outline
Python Versions
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
I. Python Basics
- A Taste of Py Mysteries
Little Programs
A Bigger Program
Python in the Real World
Python Versus the Language from Planet X
Why Python?
Why Not Python?
Python 2 Versus Python 3
Installing Python
Running Python Using the Interactive Interpreter
Using Python Files
What’s Next?
Your Moment of Zen
Coming Up
Things to Do
- Data: Types, Values, Variables, and Names Python Data Are Objects
Types
Mutability
Literal Values
Variables
Assignment
Variables Are Names, Not Places
Assigning to Multiple Names
Reassigning a Name
Copying
Choose Good Variable Names
Coming Up
Things to Do
- Numbers Booleans
Integers Literal Integers
Integer Operations
Integers and Variables
Precedence
Bases
Type Conversions
How Big Is an int?
Floats
Math Functions
Coming Up
Things to Do
- Choose with if Comment with #
Continue Lines with \
Compare with if, elif, and else
What Is True?
Do Multiple Comparisons with in
New: I Am the Walrus
Coming Up
Things to Do
- Text Strings Create with Quotes
Create with str()
Escape with \
Combine by Using +
Duplicate with *
Get a Character with []
Get a Substring with a Slice
Get Length with len()
Split with split()
Combine by Using join()
Substitute by Using replace()
Strip with strip()
Search and Select
Case
Alignment
Formatting Old style: %
New style: {} and format()
Newest Style: f-strings
More String Things
Coming Up
Things to Do
- Loop with while and for Repeat with while Cancel with break
Skip Ahead with continue
Check break Use with else
Iterate with for and in Cancel with break
Skip with continue
Check break Use with else
Generate Number Sequences with range()
Other Iterators
Coming Up
Things to Do
- Tuples and Lists Tuples Create with Commas and ()
Create with tuple()
Combine Tuples by Using +
Duplicate Items with *
Compare Tuples
Iterate with for and in
Modify a Tuple
Lists Create with []
Create or Convert with list()
Create from a String with split()
Get an Item by [ offset ]
Get Items with a Slice
Add an Item to the End with append()
Add an Item by Offset with insert()
Duplicate All Items with *
Combine Lists by Using extend() or +
Change an Item by [ offset ]
Change Items with a Slice
Delete an Item by Offset with del
Delete an Item by Value with remove()
Get an Item by Offset and Delete It with pop()
Delete All Items with clear()
Find an Item’s Offset by Value with index()
Test for a Value with in
Count Occurrences of a Value with count()
Convert a List to a String with join()
Reorder Items with sort() or sorted()
Get Length with len()
Assign with =
Copy with copy(), list(), or a Slice
Copy Everything with deepcopy()
Compare Lists
Iterate with for and in
Iterate Multiple Sequences with zip()
Create a List with a Comprehension
Lists of Lists
Tuples Versus Lists
There Are No Tuple Comprehensions
Coming Up
Things to Do
- Dictionaries and Sets Dictionaries Create with {}
Create with dict()
Convert with dict()
Add or Change an Item by [ key ]
Get an Item by [key] or with get()
Get All Keys with keys()
Get All Values with values()
Get All Key-Value Pairs with items()
Get Length with len()
Combine Dictionaries with {**a, **b}
Combine Dictionaries with update()
Delete an Item by Key with del
Get an Item by Key and Delete It with pop()
Delete All Items with clear()
Test for a Key with in
Assign with =
Copy with copy()
Copy Everything with deepcopy()
Compare Dictionaries
Iterate with for and in
Dictionary Comprehensions
Sets Create with set()
Convert with set()
Get Length with len()
Add an Item with add()
Delete an Item with remove()
Iterate with for and in
Test for a Value with in
Combinations and Operators
Set Comprehensions
Create an Immutable Set with frozenset()
Data Structures So Far
Make Bigger Data Structures
Coming Up
Things to Do
- Functions Define a Function with def
Call a Function with Parentheses
Arguments and Parameters None Is Useful
Positional Arguments
Keyword Arguments
Specify Default Parameter Values
Explode/Gather Positional Arguments with *
Explode/Gather Keyword Arguments with **
Keyword-Only Arguments
Mutable and Immutable Arguments
Docstrings
Functions Are First-Class Citizens
Inner Functions Closures
Anonymous Functions: lambda
Generators Generator Functions
Generator Comprehensions
Decorators
Namespaces and Scope
Uses of _ and __ in Names
Recursion
Async Functions
Exceptions Handle Errors with try and except
Make Your Own Exceptions
Coming Up
Things to Do
- Oh Oh: Objects and Classes What Are Objects?
Simple Objects Define a Class with class
Attributes
Methods
Initialization
Inheritance Inherit from a Parent Class
Override a Method
Add a Method
Get Help from Your Parent with super()
Multiple Inheritance
Mixins
In self Defense
Attribute Access Direct Access
Getters and Setters
Properties for Attribute Access
Properties for Computed Values
Name Mangling for Privacy
Class and Object Attributes
Method Types Instance Methods
Class Methods
Static Methods
Duck Typing
Magic Methods
Aggregation and Composition
When to Use Objects or Something Else
Named Tuples
Dataclasses
Attrs
Coming Up
Things to Do
- Modules, Packages, and Goodies Modules and the import Statement Import a Module
Import a Module with Another Name
Import Only What You Want from a Module
Packages The Module Search Path
Relative and Absolute Imports
Namespace Packages
Modules Versus Objects
Goodies in the Python Standard Library Handle Missing Keys with setdefault() and defaultdict()
Count Items with Counter()
Order by Key with OrderedDict()
Stack + Queue == deque
Iterate over Code Structures with itertools
Print Nicely with pprint()
Get Random
More Batteries: Get Other Python Code
Coming Up
Things to Do
II. Python in Practice
- Wrangle and Mangle Data Text Strings: Unicode Python 3 Unicode Strings
UTF-8
Encode
Decode
HTML Entities
Normalization
For More Information
Text Strings: Regular Expressions Find Exact Beginning Match with match()
Find First Match with search()
Find All Matches with findall()
Split at Matches with split()
Replace at Matches with sub()
Patterns: Special Characters
Patterns: Using Specifiers
Patterns: Specifying match() Output
Binary Data bytes and bytearray
Convert Binary Data with struct
Other Binary Data Tools
Convert Bytes/Strings with binascii()
Bit Operators
A Jewelry Analogy
Coming Up
Things to Do
- Calendars and Clocks Leap Year
The datetime Module
Using the time Module
Read and Write Dates and Times
All the Conversions
Alternative Modules
Coming Up
Things to Do
- Files and Directories File Input and Output Create or Open with open()
Write a Text File with print()
Write a Text File with write()
Read a Text File with read(), readline(), or readlines()
Write a Binary File with write()
Read a Binary File with read()
Close Files Automatically by Using with
Change Position with seek()
Memory Mapping
File Operations Check Existence with exists()
Check Type with isfile()
Copy with copy()
Change Name with rename()
Link with link() or symlink()
Change Permissions with chmod()
Change Ownership with chown()
Delete a File with remove()
Directory Operations Create with mkdir()
Delete with rmdir()
List Contents with listdir()
Change Current Directory with chdir()
List Matching Files with glob()
Pathnames Get a Pathname with abspath()
Get a symlink Pathname with realpath()
Build a Pathname with os.path.join()
Use pathlib
BytesIO and StringIO
Coming Up
Things to Do
- Data in Time: Processes and Concurrency Programs and Processes Create a Process with subprocess
Create a Process with multiprocessing
Kill a Process with terminate()
Get System Info with os
Get Process Info with psutil
Command Automation Invoke
Other Command Helpers
Concurrency Queues
Processes
Threads
concurrent.futures
Green Threads and gevent
twisted
asyncio
Redis
Beyond Queues
Coming Up
Things to Do
- Data in a Box: Persistent Storage Flat Text Files
Padded Text Files
Tabular Text Files CSV
XML
An XML Security Note
HTML
JSON
YAML
Tablib
Pandas
Configuration Files
Binary Files Padded Binary Files and Memory Mapping
Spreadsheets
HDF5
TileDB
Relational Databases SQL
DB-API
SQLite
MySQL
PostgreSQL
SQLAlchemy
Other Database Access Packages
NoSQL Data Stores The dbm Family
Memcached
Redis
Document Databases
Time Series Databases
Graph Databases
Other NoSQL
Full-Text Databases
Coming Up
Things to Do
- Data in Space: Networks TCP/IP Sockets
Scapy
Netcat
Networking Patterns
The Request-Reply Pattern ZeroMQ
Other Messaging Tools
The Publish-Subscribe Pattern Redis
ZeroMQ
Other Pub-Sub Tools
Internet Services Domain Name System
Python Email Modules
Other Protocols
Web Services and APIs
Data Serialization Serialize with pickle
Other Serialization Formats
Remote Procedure Calls XML RPC
JSON RPC
MessagePack RPC
Zerorpc
gRPC
Twirp
Remote Management Tools
Big Fat Data Hadoop
Spark
Disco
Dask
Clouds Amazon Web Services
Google Cloud
Microsoft Azure
OpenStack
Docker Kubernetes
Coming Up
Things to Do
- The Web, Untangled Web Clients Test with telnet
Test with curl
Test with httpie
Test with httpbin
Python’s Standard Web Libraries
Beyond the Standard Library: requests
Web Servers The Simplest Python Web Server
Web Server Gateway Interface (WSGI)
ASGI
Apache
NGINX
Other Python Web Servers
Web Server Frameworks Bottle
Flask
Django
Other Frameworks
Database Frameworks
Web Services and Automation webbrowser
webview
Web APIs and REST
Crawl and Scrape Scrapy
BeautifulSoup
Requests-HTML
Let’s Watch a Movie
Coming Up
Things to Do
- Be a Pythonista About Programming
Find Python Code
Install Packages Use pip
Use virtualenv
Use pipenv
Use a Package Manager
Install from Source
Integrated Development Environments IDLE
PyCharm
IPython
Jupyter Notebook
JupyterLab
Name and Document
Add Type Hints
Test Check with pylint, pyflakes, flake8, or pep8
Test with unittest
Test with doctest
Test with nose
Other Test Frameworks
Continuous Integration
Debug Python Code Use print()
Use Decorators
Use pdb
Use breakpoint()
Log Error Messages
Optimize Measure Timing
Algorithms and Data Structures
Cython, NumPy, and C Extensions
PyPy
Numba
Source Control Mercurial
Git
Distribute Your Programs
Clone This Book
How You Can Learn More Books
Websites
Groups
Conferences
Getting a Python Job
Coming Up
Things to Do
- Py Art 2-D Graphics Standard Library
PIL and Pillow
ImageMagick
3-D Graphics
3-D Animation
Graphical User Interfaces
Plots, Graphs, and Visualization Matplotlib
Seaborn
Bokeh
Games
Audio and Music
Coming Up
Things to Do
- Py at Work The Microsoft Office Suite
Carrying Out Business Tasks
Processing Business Data Extracting, Transforming, and Loading
Data Validation
Additional Sources of Information
Open Source Python Business Packages
Python in Finance
Business Data Security
Maps Formats
Draw a Map from a Shapefile
Geopandas
Other Mapping Packages
Applications and Data
Coming Up
Things to Do
- Py Sci Math and Statistics in the Standard Library Math Functions
Working with Complex Numbers
Calculate Accurate Floating Point with decimal
Perform Rational Arithmetic with fractions
Use Packed Sequences with array
Handling Simple Stats with statistics
Matrix Multiplication
Scientific Python
NumPy Make an Array with array()
Make an Array with arange()
Make an Array with zeros(), ones(), or random()
Change an Array’s Shape with reshape()
Get an Element with []
Array Math
Linear Algebra
SciPy
SciKit
Pandas
Python and Scientific Areas
Coming Up
Things to Do
A. Hardware and Software for Beginning Programmers Hardware Caveman Computers
Electricity
Inventions
An Idealized Computer
The CPU
Memory and Caches
Storage
Inputs
Outputs
Relative Access Times
Software In the Beginning Was the Bit
Machine Language
Assembler
Higher-Level Languages
Operating Systems
Virtual Machines
Containers
Distributed Computing and Networks
The Cloud
Kubernetes
B. Install Python 3 Check Your Python Version
Install Standard Python macOS
Windows
Linux or Unix
Install the pip Package Manager
Install virtualenv
Other Packaging Solutions
Install Anaconda Install Anaconda’s Package Manager conda
C. Something Completely Different: Async Coroutines and Event Loops
Asyncio Alternatives
Async Versus…
Async Frameworks and Servers
D. Answers to Exercises 1. A Taste of Py
- Data: Types, Values, Variables, and Names
- Numbers
- Choose with if
- Text Strings
- Loop with while and for
- Tuples and Lists
- Dictionaries
- Functions
- Oh Oh: Objects and Classes
- Modules, Packages, and Goodies
- Wrangle and Mangle Data
- Calendars and Clocks
- Files and Directories
- Data in Time: Processes and Concurrency
- Data in a Box: Persistent Storage
- Data in Space: Networks
- The Web, Untangled
- Be a Pythonista
- Py Art
- Py at Work
- PySci
E. Cheat Sheets Operator Precedence
String Methods Change Case
Search
Modify
Format
String Type
String Module Attributes
Coda
Index
Preface:
“As the title promises, this book will introduce you to one of the world’s most popular programming languages: Python. It’s aimed at beginning programmers as well as more experienced programmers who want to add Python to the languages they already know.” (IPyBLub)
“In most cases, it’s easier to learn a computer language than a human language. There’s less ambiguity and fewer exceptions to keep in your head. Python is one of the most consistent and clear computer languages. It balances ease of learning, ease of use, and expressive power.” (IPyBLub)
“Computer languages are made of data (like nouns in spoken languages) and instructions or code (like verbs). You need both. In alternating chapters, you’ll be introduced to Python’s basic code and data structures, learn how to combine them, and build up to more advanced ones. The programs that you read and write will get longer and more complex. Using a woodworking analogy, we’ll start with a hammer, nails, and scraps of wood. Over the first half of this book, we’ll introduce more specialized components, up to the equivalents of lathes and other power tools.” (IPyBLub)
“You’ll not only learn the language, but also what to do with it. We’ll begin with the Python language and its “batteries included” standard library, but I’ll also show you how to find, download, install, and use some good third-party packages. My emphasis is on whatever I’ve actually found useful in more than 10 years of production Python development, rather than fringe topics or complex hacks.” (IPyBLub)
“Although this is an introduction, some advanced topics are included because I want to expose them to you. Areas like databases and the web are still covered, but technology changes fast. A Python programmer might now be expected to know something about cloud computing, machine learning, or event streaming. You’ll find something here on all of these.” (IPyBLub)
“Python has some special features that work better than adapting styles from other languages that you may know. For example, using for and iterators is a more direct way of making a loop than manually incrementing some counter variable.” (IPyBLub)
“When you’re learning something new, it’s hard to tell which terms are specific instead of colloquial, and which concepts are actually important. In other words, “Is this on the test?” I’ll highlight terms and ideas that have specific meaning or importance in Python, but not too many at once. Real Python code is included early and often.” (IPyBLub)
Note: “I’ll include a note such as this when something might be confusing, or if there’s a more appropriate Pythonic way to do it.” (IPyBLub)
“Python isn’t perfect. I’ll show you things that seem odd or that should be avoided — and offer alternatives you can use, instead.” (IPyBLub)
“Now and then, my opinions on some subjects (such as object inheritance, or MVC and REST designs for the web) may vary a bit from the common wisdom. See what you think.” (IPyBLub)
Audience:
“This book is for anybody interested in learning one of the world’s most popular computing languages, regardless of whether you have previously learned any programming.” (IPyBLub)
Changes in the Second Edition:
“What’s changed since the first edition?
About a hundred more pages, including cat pictures.
Twice the chapters, each shorter now.
An early chapter devoted to data types, variables, and names.
New standard Python features like f-strings.
New or improved third-party packages.
New code examples throughout.
An appendix on basic hardware and software, for new programmers.
An appendix on asyncio, for not-so-new programmers.
“New stack” coverage: containers, clouds, data science, and machine learning.
Hints on getting a job programming in Python.
What hasn’t changed? Examples using bad poetry and ducks. These are evergreen.” (IPyBLub)
Outline:
“Part I (Chapters 1–11) explains Python’s basics. You should read these chapters in order. I work up from the simplest data and code structures, combining them on the way into more detailed and realistic programs. Part II (Chapters 12–22) shows how Python is used in specific application areas such as the web, databases, networks, and so on; read these chapters in any order you like.” (IPyBLub)
“Here’s a brief preview of the chapters and appendixes, including some of the terms that you’ll run into there:” (IPyBLub)
Chapter 1, A Taste of Py
“Computer programs are not that different from directions that you see every day. Some little Python programs give you a glimpse of the language’s looks, capabilities, and uses in the real world. You’ll see how to run a Python program within its interactive interpreter (or shell), or from a text file saved on your computer.” (IPyBLub)
Chapter 2, Data: Types, Values, Variables, and Names
“Computer languages mix data and instructions. Different types of data are stored and treated differently by the computer. They may allow their values to be changed (mutable) or not (immutable). In a Python program, data can be literal (numbers like 78, text strings like “waffle”) or represented by named variables. Python treats variables like names, which is different from many other languages and has some important consequences.” (IPyBLub)
Chapter 3, Numbers
“This chapter shows Python’s simplest data types: booleans, integers, and floating-point numbers. You’ll also learn the basic math operations. The examples use Python’s interactive interpreter like a calculator.” (IPyBLub)
Chapter 4, Choose with if
“We’ll bounce between Python’s nouns (data types) and verbs (program structures) for a few chapters. Python code normally runs a line at a time, from the start to the end of a program. The if code structure lets you run different lines of code, depending on some data comparison.” (IPyBLub)
Chapter 5, Text Strings
“Back to nouns, and the world of text strings. Learn how to create, combine, change, retrieve, and print strings.” (IPyBLub)
Chapter 6, Loop with while and for
“Verbs again, and two ways to make a loop: for and while. You’ll be introduced to a core Python concept: iterators.” (IPyBLub)
Chapter 7, Tuples and Lists
“It’s time for the first of Python’s higher-level built-in data structures: lists and tuples. These are sequences of values, like LEGO for building much more complex data structures. Step through them with iterators, and build lists quickly with comprehensions.” (IPyBLub)
Chapter 8, Dictionaries and Sets
“Dictionaries (aka dicts) and sets let you save data by their values rather than their position. This turns out to be very handy and will be among your favorite Python features.” (IPyBLub)
Chapter 9, Functions
“Weave the data and code structures of the previous chapters to compare, choose, or repeat. Package code in functions and handle errors with exceptions.” (IPyBLub)
Chapter 10, Oh Oh: Objects and Classes
“The word object is a bit fuzzy, but important in many computer languages, including Python. If you’ve done object-oriented programming in other languages, Python is a bit more relaxed. This chapter explains how to use objects and classes, and when it’s better to use alternatives.” (IPyBLub)
Chapter 11, Modules, Packages, and Goodies
“This chapter demonstrates how to scale out to larger code structures: modules, packages, and programs. You’ll see where to put code and data, how to get data in and out, handle options, tour the Python Standard Library, and take a glance at what lies beyond.” (IPyBLub)
Chapter 12, Wrangle and Mangle Data
“Learn to manage (or mangle) data like a pro. This chapter is all about text and binary data, joy with Unicode characters, and regex text searching. It also introduces the data types bytes and bytearray, counterparts of strings that contain raw binary values instead of text characters.” (IPyBLub)
Chapter 13, Calendars and Clocks
“Dates and times can be messy to handle. This chapter shows common problems and useful solutions.” (IPyBLub)
Chapter 14, Files and Directories
“Basic data storage uses files and directories. This chapter shows you how to create and use them.” (IPyBLub)
Chapter 15, Data in Time: Processes and Concurrency
“This is the first hard-core system chapter. Its theme is data in time — how to use programs, processes, and threads to do more things at a time (concurrency). Python’s recent async additions are mentioned, with details in Appendix C.” (IPyBLub)
Chapter 16, Data in a Box: Persistent Storage
“Data can be stored and retrieved with basic flat files and directories within filesystems. They gain some structure with common text formats such as CSV, JSON, and XML. As data get larger and more complex, they need the services of databases — traditional relational ones, and some newer NoSQL data stores.” (IPyBLub)
Chapter 17, Data in Space: Networks
“Send your code and data through space in networks with services, protocols, and APIs. Examples range from low-level TCP sockets, to messaging libraries and queuing systems, to cloud deployment.” (IPyBLub)
Chapter 18, The Web, Untangled
“The web gets its own chapter — clients, servers, APIs, and frameworks. You’ll crawl and scrape websites, and then build real websites with request parameters and templates.” (IPyBLub)
Chapter 19, Be a Pythonista
“This chapter contains tips for Python developers — installation with pip and virtualenv, using IDEs, testing, debugging, logging, source control, and documentation. It also helps you to find and install useful third-party packages, package your own code for reuse, and learn where to get more information.” (IPyBLub)
Chapter 20, Py Art
“People are doing cool things with Python in the arts: graphics, music, animation, and games.” (IPyBLub)
Chapter 21, Py at Work
“Python has specific applications for business: data visualization (plots, graphs, and maps), security, and regulation.” (IPyBLub)
Chapter 22, Py Sci
“In the past few years, Python has emerged as a top language for science: math and statistics, physical science, bioscience, and medicine. Data science and machine learning are notable strengths. This chapter touches on NumPy, SciPy, and Pandas.” (IPyBLub)
Appendix A, Hardware and Software for Beginning Programmers
“If you’re fairly new to programming, this describes how hardware and software actually work. It introduces some terms that you’ll keep running into.” (IPyBLub)
Appendix B, Install Python 3
“If you don’t already have Python 3 on your computer, this appendix shows you how to install it, whether you’re running Windows, macOS, Linux, or some other variant of Unix.” (IPyBLub)
Appendix C, Something Completely Different: Async
“Python has been adding asynchronous features in different releases, and they’re not easy to understand. I mention them as they come up in various chapters, but save a detailed discussion for this appendix.” (IPyBLub)
Appendix D, Answers to Exercises
“This has the answers to the end-of-chapter exercises. Don’t peek here until you’ve tried the exercises yourself, or you might be turned into a newt.” (IPyBLub)
Appendix E, Cheat Sheets
“This appendix contains cheat sheets to use as a quick reference.” (IPyBLub)