Course goals: design algorithms, analyze algorithms, fundamental algorithms, math techniques in support of analysis. Research techniques.
Models of computation: uniform-cost RAM model vs. logarithmic-cost RAM mode. PRAM, hypercube, fat tree, mesh. Quantum computation and Peter Schnor's algorithm.
Criteria for evaluating and comparing algorithms: Resource usage (time, space, communication, energy), quality of results (correctness, performance ratio, special properties), and implementation difficulty (man hours, programming cost in dollars, lines of code, Kolmogorov complexity).
Overview of analysis techniques: analyzing straight-line programs, looping programs, and recursive programs. Setting up and solving summations and recurrences. Probabilistic and amortized analysis. Systems of recurrence relations. Hints for solving HW 1.
Motivation for Problem 5 of HW1: Bennett's algorithm for converting any program into an equivalent reversible program. Thermodynamics and computation: Landauer's principle (energy dissipation proportional to kT(ln 2)n, where k is Boltzman's constant, T is the temperature of the computer, and n is the number bits of information lost in the computation.)
Project overview: Cycle detection. Floyd's 2-finger algorithm and its improvement by a time-space tradeoff (Sedgewick et. al.) Applications.
Criteria for evaluating journal submissions: appropriateness, scientific merit (correctness, novelty, non-triviality, significance), and effective presentation. These are the criteria which will be applied to the project report.
What is running time? Worst case versus average case. Orders of growth. Common orders of growth. Constants, polylogarithms, polynomials, superpolynomial-subexponentials, exponentials. Superpolylogarithmic Subexponential Functions.
Asymptotic notations: O, o, Omega, omega, Theta, ~. Lower bounds vs. upper bounds. Worst-case vs. average case. How to compare growth rates (def, ration test, numerical, intuition). Ratio test for comparing functions. Modern vs. old approach to big-Oh notation.
How to analyze algorithms. Straight-line algorithms, looping algorithms, and recursive algorithms. Examples: insertion-sort, mergesort, Strassen's algorithm. Relationships between design and analysis. Divide-and-conqurer.
Summations to recurrences, and vice-versa. Solving recurrences (iteration, guess and check, master theorem, generating functions, characteristic equations, transformation, bounding). Examples. See Gilles Brassard (Algorithmics) for characteristic equations, and Fred Roberts (Applied Combinatorics) for generating functions.
"Algorithm's Policy" (read policy, including description, I/O specs, correctness, analysis).
Matroids, generic greedy algorithm (and its proof of correctness), and examples (e.g. MST). Practical applications: how to design greedy algorithms and prove their correctness by identifying the underlying matroid explicitly.
Also: (1) Discussion of cost of merging two \sqrt{n} length sorted lists. (2) Dynamic programming in chess endings (Thompson, and Stiller). (3) Professional societies and the "great crib sheet for theoretical computer science" from SIGACT News.
Review the Floyd/Warshall algorithm and generalize it to closed semirings. Motivate the various properties of closed semirings. Rediscover the generic abstracted algorithm as an example of dynamic programming. Note the issue of traversing cycles zero or more times. Apply to selected applications, including shortest paths, transitive closure, path expressions, regular expressions for finite state automata, and solving systems of linear equations (Gaussian elimination).
Practice recurrences for Exam I. Understand what's important in a recurrence and what's not. Understand the difference between asympototic domination vs. asymptotic polynomial domination. Know the generalized Case #2 of the Master Theorem. Understand how to deal with characteristic equations with roots of multipilicy greater than one.
Questions and answers.
Three methods for doing amortized analysis: aggregate method, accountant's method, and potential function method. Toy example with stacks. On-line vs. off-line algorithms.
Lectures 1-8. Does not include Lecture 9.
Be prepared to: Design and analyze looping and recursive algorithms, including mutually recursive programs. Design algorithms based on basic design strategies, including dynamic programming. Solve summations and recurrences, including by the methods of transformation, characteristic equations, and guess and check. Apply asymptotic notations. Apply the theory of matroids to design and analyze greedy algorithms. Apply the theory of closed semirings to apply path algorithms to general settings. Be familair with fundamental algorithms, including simmulated annealing.