Arnold, Taylor

A computational approach to statistical learning - Boca Raton CRC Press 2019 - xiii, 361 p. Includes bibliographical references and index - Chapman & hall/ CRC texts in statistical science .

Table of contents:

1. Introduction

Computational approach

Statistical learning



How to read this book

Supplementary materials

Formalisms and terminology


2. Linear Models


Ordinary least squares

The normal equations

Solving least squares with the singular value decomposition

Directly solving the linear system

(*) Solving linear models with orthogonal projection

(*) Sensitivity analysis

(*) Relationship between numerical and statistical error

Implementation and notes

Application: Cancer incidence rates


3. Ridge Regression and Principal Component Analysis

Variance in OLS

Ridge regression

(*) A Bayesian perspective

Principal component analysis

Implementation and notes

Application: NYC taxicab data


4. Linear Smoothers


Basis expansion

Kernel regression

Local regression

Regression splines

(*) Smoothing splines

(*) B-splines

Implementation and notes

Application: US census tract data


5. Generalized Linear Models

Classification with linear models

Exponential families

Iteratively reweighted GLMs

(*) Numerical issues

(*) Multi-class regression

Implementation and notes

Application: Chicago crime prediction


6. Additive Models

Multivariate linear smoothers

Curse of dimensionality

Additive models

(*) Additive models as linear models

(*) Standard errors in additive models

Implementation and notes

Application: NYC flights data


7. Penalized Regression Models

Variable selection

Penalized regression with the `- and `-norms

Orthogonal data matrix

Convex optimization and the elastic net

Coordinate descent

(*) Active set screening using the KKT conditions

(*) The generalized elastic net model

Implementation and notes

Application: Amazon product reviews


8. Neural Networks

Dense neural network architecture

Stochastic gradient descent

Backward propagation of errors

Implementing backpropagation

Recognizing hand written digits

(*) Improving SGD and regularization

(*) Classification with neural networks

(*) Convolutional neural networks

Implementation and notes

Application: Image classification with EMNIST


9. Dimensionality Reduction

Unsupervised learning

Kernel functions

Kernel principal component analysis

Spectral clustering

t-Distributed stochastic neighbor embedding (t-SNE)


Implementation and notes

Application: Classifying and visualizing fashion MNIST


10. Computation in Practice

Reference implementations

Sparse matrices

Sparse generalized linear models

Computation on row chunks

Feature hashing

Data quality issues

Implementation and notes



A Matrix Algebra

A Vector spaces

A Matrices

A Other useful matrix decompositions

B Floating Point Arithmetic and Numerical Computation

B Floating point arithmetic

B Numerical sources of error

B Computational effort

A Computational Approach to Statistical Learning gives a novel introduction to predictive modeling by focusing on the algorithmic and numeric motivations behind popular statistical methods. The text contains annotated code to over 80 original reference functions. These functions provide minimal working implementations of common statistical learning algorithms. Every chapter concludes with a fully worked out application that illustrates predictive modeling tasks using a real-world dataset.

The text begins with a detailed analysis of linear models and ordinary least squares. Subsequent chapters explore extensions such as ridge regression, generalized linear models, and additive models. The second half focuses on the use of general-purpose algorithms for convex optimization and their application to tasks in statistical learning. Models covered include the elastic net, dense neural networks, convolutional neural networks (CNNs), and spectral clustering. A unifying theme throughout the text is the use of optimization theory in the description of predictive models, with a particular focus on the singular value decomposition (SVD). Through this theme, the computational approach motivates and clarifies the relationships between various predictive models.


Machine learning - Mathematics
Mathematical statistics
Estimation theory

006.31015195 / A7C6

Powered by Koha