A modern approach to AI · Appendix A

Mathematical Background

A compact refresher on growth rates, linear algebra, and probability—the mathematical tools used throughout AI.

Appendix guide

  1. A.1 Complexity Analysis and O() Notation

    Big-O describes an asymptotic upper-growth bound, suppressing constants and lower-order terms. It helps compare resource scaling, not exact runtime.

  2. A.2 Vectors, Matrices, and Linear Algebra

    Vectors represent ordered quantities; matrices represent linear transformations. Dot products, matrix products, and norms support geometry and computation.

  3. A.3 Probability Distributions

    A distribution assigns probabilities to outcomes. Conditional probability updates beliefs given evidence; expectations summarize average outcomes.

Bibliographical and Historical Notes accompany the appendix in the book.

Interactive: matrix–vector product

Change vector components to compute y = Ax. Each output is a row–vector dot product.

A = [[2, 1], [−1, 3]]

21−13

Input x

x = [, ]

Output y = Ax

[, ]

Reading the math

Big-O describes how resource use grows with input size, ignoring constant factors; it is not a stopwatch prediction. A vector is an ordered list of components, a matrix is a rectangular array representing a linear map, and a dot product multiplies paired components and sums them. For the demo's A=[[2,1],[−1,3]] and x=[2,1], the first row gives 2·2+1·1=5; the second gives −1·2+3·1=1. A probability distribution assigns nonnegative weights summing to one; an expectation is the probability-weighted average outcome.

The sliders use integer components between −5 and 5, so the example is exact arithmetic with no rounding or numerical instability.

Matrix–vector complexity

For a dense m×n matrix and n-component input, each of m output rows uses n multiply-adds, so y=Ax takes O(mn) time. Storing the output uses O(m) space; with the output excluded, the straightforward row-at-a-time calculation needs O(1) auxiliary workspace. If the full input matrix and vector are counted as resident data, they occupy O(mn+n) storage. This interactive example fixes m=n=2 and directly evaluates two formulas, so each slider update is O(1) time and auxiliary space; its fixed controls/output are O(1) DOM memory too.