Math for Intelligent Systems, COT5615

COT 5615, Math for Intelligent Systems, Spring 2020

Place:CSE; A101
Time:Tuesday 8,9 (3:00-4:55 p.m.), Thursday 9 (4:05-4:55 p.m.)

Instructor:
Arunava Banerjee
Office: CSE E336.
E-mail: arunava@cise.ufl.edu.
Phone: 505-1556.
Office hours: Wednesday 2:00 p.m.-4:00 p.m. or by appointment.

TA:
XXX
TA Office: CSE E309.
Office hours: XXXX p.m.

Catalog Description:

COT 5615: Mathematics for Intelligent Systems

Credits: 3 Grading Scheme: Letter

Prerequisite: MAC 2313, Multivariate Calculus; MAS 3114 or MAS 4105, Linear Algebra; STA 4321, Mathematical Statistics.

Mathematical methods commonly used to develop algorithms for computer systems that exhibit intelligent behavior.

Course Objectives:

The goal of this course is to cover several topics in mathematics that is of general interest to people pursuing a Ph.d in intelligent systems. The course will focus on conceptual clarity.

This course is an official pre-requisite for CAP6610 (Machine learning)

Required Text:

There is no official text book for this course. We will mostly work with material posted online. However, following are four good books to have.

References:
Principles of Mathematical Analysis, W. Rudin
Linear Algebra, K. M. Hoffman, R. Kunze
Probability and Measure, P. Billingsley
Probability: Theory and examples, R. Durrett, can be found online at http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.155.4899

Please return to this page at least once a week to check updates in the table below

Evaluation: The final grade will be based on three midterm exams (25% each) and several assignments (remaining 25%).

Course Policies:

Late assignments: All homework assignments are due before class.
Plagiarism: You are expected to submit your own solutions to the assignments. Feel free to discuss the concepts underlying the questions with friends and classmates.
Attendance: Their is no official attendance requirement. If you find better use of the time spent sitting thru lectures, please feel free to devote such to any occupation of your liking. However, keep in mind that it is your responsibility to stay abreast of the material presented in class.
Cell Phones: Please, no phone calls during class. Please turn off the ringer on your cell phone before coming to class.
University Honesty Policy, Campus Resources and other important information : See here for grad and here for undergrad.

Tentative List of Topics to be covered

Real analysis: Rationals and Reals, Basic point set topology, convergence, continuity etc (approximately Chap 1-4 of Rudin)
Vector spaces and Linear algebra (approximately Chap 1-3, 8 of Hoff. Kun.)
Mathematical Probability theory: Sigma algebra, Random variables etc. (approximately Chap 1. of Durett)
Information Theory: Entropy, Mutual Information, etc.

Important Announcement
Exam Schedule

Midterm I will be held on Feb 04th in class.
Midterm II will be held on Mar 17th in class.
Midterm III will be held on Apr 21st in class.
Each exam will last 2 hrs.
1 letter sized cheat sheat (both sides) allowed. Closed Book, closed notes.
There will be no final exam.

List of Topics covered

Week Topic Additional Reading Assignment

Jan 05 - Jan 11

Introduction
How to prove things
Proof of the Fundamental Theorem of Arithmetic (unique factorization)
Proof of infintely many primes
Twin primes conjecture
Johnson Lindenstrauss lemma
Robin's Theorem about the Riemann Hypothesis
P versus NP

Paper on the Riemann Hypothesis
Paper on the Johnson Lindenstrauss lemma

Jan 12 - Jan 18

Peano Axioms
Field axioms
Natural numbers, Integers
Abstract formulation of Rational Numbers
Proof of Pythogorus Theorem
Sqrt(2) is not rational
Partial and Total order
Least upper bound (Supremum) and Greatest lower bound (infimum)
Least upper bound property.
Proof: Rationals do not have the LUB property

Peano Axioms
Zermelo Frankel Set Theory

Jan 19 - Jan 25

Theorem: LUB property IFF GLB property.
Reals as Dedekind cuts
Fields: The rational and real fields
Functions; injective, surjective and bijective
Cardinality of Integers, Rationals and Reals
Countably infinite (proof for rationals)
Reals are uncountable (Cantor's diagonalization)
Cardinality of set versus Power set (Cantor's diagonalization)

More on Dedekind Cuts

Jan 26 - Feb 01

Convergent sequence, Cauchy sequence
Theorem: Every convergent sequence is a Cauchy sequence
Metric spaces
epsilon neighborhoods, limit points, interior points
Open sets, Closed sets
Thm: epsilon neighborhood is open
Thm: Arbitrary union, Finite intersection of open sets is open
Thm: Open iff complement is closed
Thm: Arbitrary intersection, Finite union of closed sets is closed
Definition: Topological space.

Feb 02 - Feb 08

MIDTERM I
Complete Metric space; Seperable Metric Space
Definition: Topological space, trivial topology, discrete topology
Metrizable Topological Space.

Feb 09 - Feb 15

Compact sets; properties and theorems.
Thm: compact set is closed. Closed subset of compact set is compact.
Stmt of Heine–Borel theorem
Continuity: Limit definition of continuous functions
Weierstrass's definition of continuous fns
Proof of equivalance
Pointwise continuity, Uniform continuity
Dirichlet function, Thomae's function
Compactness and continuity
C[a,b] as a metric space

Feb 16 - Feb 22

Pointwise convergence, Uniform convergence of functions
Lagrange Polynomials and interpolation
Bernstein Polynomials; properties.
Weierstrass Approximation theorem

Proof of the WAT
Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.

Feb 23 - Feb 29

Finished proof of WAT
Linear/Vector spaces; Linear maps; examples
Definition of Vector/Linear space on a Field.

Mar 01 - Mar 07

SPRING BREAK

Mar 08 - Mar 14

Lagrange Polynomials
Subspace, Span of a set of vectors

Mar 15 - Mar 21

Linear Independence, Basis
Matrix representation of a linear map
Matrix multiplication with vector and relationship to linear maps
Linear algebra of linear transforms
Change of basis as a linear transform
Worked out example of Lagrange Polynomials as a basis

Mar 22 - Mar 28

Image deblurring: convolution is a linear operator
Matrix inverse
Matrix product and relationship to Composition
Gaussian elimination
LU decomposition

Mar 29 - Apr 04

Rank Nullity Theorem
Normed (+Complete=Banach) vector spaces, Inner product spaces
Lp Norms
Inner product (+Complete=Hilbert) spaces
Induced norm.
Orthonormal basis; advantages, orthonormal vectors are independent.
Gram Schmidt orthogonalization
QR Decomposition

Apr 05 - Apr 11

Fourier series
Riesz–Fischer theorem (w/o proof)
Eigenvectors/eigenvalues
Real symmetric matrices have real eigen values and orthogonal eigen vectors (Proof).

Apr 12 - Apr 19

Mathematical Probability Theory
Probability Space: Sample space, outcome, sigma-algebra of events
Random Variables, Indicator Random variables
Probability distribution function
MIDTERM II

Week	Topic	Additional Reading	Assignment
Jan 05 - Jan 11	Introduction How to prove things Proof of the Fundamental Theorem of Arithmetic (unique factorization) Proof of infintely many primes Twin primes conjecture Johnson Lindenstrauss lemma Robin's Theorem about the Riemann Hypothesis P versus NP	Paper on the Riemann Hypothesis Paper on the Johnson Lindenstrauss lemma
Jan 12 - Jan 18	Peano Axioms Field axioms Natural numbers, Integers Abstract formulation of Rational Numbers Proof of Pythogorus Theorem Sqrt(2) is not rational Partial and Total order Least upper bound (Supremum) and Greatest lower bound (infimum) Least upper bound property. Proof: Rationals do not have the LUB property	Peano Axioms Zermelo Frankel Set Theory
Jan 19 - Jan 25	Theorem: LUB property IFF GLB property. Reals as Dedekind cuts Fields: The rational and real fields Functions; injective, surjective and bijective Cardinality of Integers, Rationals and Reals Countably infinite (proof for rationals) Reals are uncountable (Cantor's diagonalization) Cardinality of set versus Power set (Cantor's diagonalization)	More on Dedekind Cuts
Jan 26 - Feb 01	Convergent sequence, Cauchy sequence Theorem: Every convergent sequence is a Cauchy sequence Metric spaces epsilon neighborhoods, limit points, interior points Open sets, Closed sets Thm: epsilon neighborhood is open Thm: Arbitrary union, Finite intersection of open sets is open Thm: Open iff complement is closed Thm: Arbitrary intersection, Finite union of closed sets is closed Definition: Topological space.
Feb 02 - Feb 08	MIDTERM I Complete Metric space; Seperable Metric Space Definition: Topological space, trivial topology, discrete topology Metrizable Topological Space.
Feb 09 - Feb 15	Compact sets; properties and theorems. Thm: compact set is closed. Closed subset of compact set is compact. Stmt of Heine–Borel theorem Continuity: Limit definition of continuous functions Weierstrass's definition of continuous fns Proof of equivalance Pointwise continuity, Uniform continuity Dirichlet function, Thomae's function Compactness and continuity C[a,b] as a metric space
Feb 16 - Feb 22	Pointwise convergence, Uniform convergence of functions Lagrange Polynomials and interpolation Bernstein Polynomials; properties. Weierstrass Approximation theorem	Proof of the WAT Note that there are typos on page 4. Should be K2 instead of K1 in the first two inequalities.
Feb 23 - Feb 29	Finished proof of WAT Linear/Vector spaces; Linear maps; examples Definition of Vector/Linear space on a Field.
Mar 01 - Mar 07	SPRING BREAK
Mar 08 - Mar 14	Lagrange Polynomials Subspace, Span of a set of vectors
Mar 15 - Mar 21	Linear Independence, Basis Matrix representation of a linear map Matrix multiplication with vector and relationship to linear maps Linear algebra of linear transforms Change of basis as a linear transform Worked out example of Lagrange Polynomials as a basis
Mar 22 - Mar 28	Image deblurring: convolution is a linear operator Matrix inverse Matrix product and relationship to Composition Gaussian elimination LU decomposition
Mar 29 - Apr 04	Rank Nullity Theorem Normed (+Complete=Banach) vector spaces, Inner product spaces Lp Norms Inner product (+Complete=Hilbert) spaces Induced norm. Orthonormal basis; advantages, orthonormal vectors are independent. Gram Schmidt orthogonalization QR Decomposition
Apr 05 - Apr 11	Fourier series Riesz–Fischer theorem (w/o proof) Eigenvectors/eigenvalues Real symmetric matrices have real eigen values and orthogonal eigen vectors (Proof).
Apr 12 - Apr 19	Mathematical Probability Theory Probability Space: Sample space, outcome, sigma-algebra of events Random Variables, Indicator Random variables Probability distribution function MIDTERM II