Description:
Statistical learning theory is a burgeoning research field at the intersection
of probability, statistics, computer science, and optimization that studies the performance
of computer algorithms for making predictions on the basis of training data.
The following topics will be covered: basics of statistical decision theory;
concentration inequalities; supervised and unsupervised learning; empirical risk minimization;
complexity-regularized estimation; generalization bounds for learning algorithms;
VC dimension and Rademacher complexities; minimax lower bounds; online learning
and optimization. Along with the general theory, we will discuss a number of
applications of statistical learning theory to signal processing, information
theory, and adaptive control.
Problem sets:
Exams:
Here is a link to Spring 2017 Student Projects
Grading scheme:
Homework (40%), two ninety minute midterms (20% each), project (20%).
Prerequisite:
ECE 534, Random Processes
Credit: 4 graduate hours
Lecture times and location:
TuTh 2:00-3:20 p.m. in Room 2015 ECE Building
Assigned Reading:
The reading will mainly be from
notes prepared for this course by
Prof. Max Raginsky .
See the
Fall 2013,
Fall 2014, or
Fall 2015
websites for earlier versions of the notes and related references. Additional
reading may be assigned from other books or articles, including:
Course Staff and Office Hours:
Question and answer site:
Piazza
About the project:
For the project you are to choose a topic related to the course content and understand and critically evaluate
two or three major papers in that area. Then demonstrate knowledge of the papers by working an example
based on a paper or possibly extending the theory of a paper. You will need to write a project report of five
to ten pages in length, and prepare a fifteen minute presentation.
Additional policy:
Collaboration on the homework is permitted, however each student must write and submit independent solutions. Homework is due within the first 5 minutes of the class period on the due date. No late homework will be accepted (unless an extension is granted in advance by the instructor).
Problem set 1 solutions .tex notes
Problem set 2 solutions .tex notes
Problem set 3 solutions .tex notes
Problem set 4 solutions .tex notes
Problem set 5 solutions .tex notes
Problem set 6 solutions .tex notes
Problem set 7 solutions .tex notes
Exam 1 solutions
Exam 2 solutions
Here is exam 1 study guide and a
summary of parts V-VII of the course.
Project presentations will be scheduled MWF, May 8-12, 10am-12:20pm. Please select a time slot and enter topic
(when you know it) in the Wiki under the tools section of the Compass page for the course.
Project write-ups are due 5pm Friday, May 12.
Bruce Hajek, Instructor
b-hajek AT illinois dot edu Office Hours:
Wednesdays, 1-3 pm in Room 105 CSL
Harsh Gupta, TA
hgupta10 AT illinois dot edu Office Hours:
Wednesdays, 11 am - 1 pm in Room 2036 ECEB
Muhammed Sayin, TA
sayin2 AT illinois dot edu
Office Hours:
Mondays, 2-4 pm in Room 2036 ECEB
Amir Taghvaei, TA
taghvae2 AT illinois dot edu Office Hours:
Mondays 4-6 pm in Room 2036 ECEB
Optional recitation sessions.
Staffed by TAs. Fridays, 1pm-2pm in Room 3020 ECEB (north tower)
TAs will work out
sample problems and examples, review background material as needed.
You are encouraged to do
your homework in Latex. The .tex source of the problem set is provided for your convenience. Also, you can upload
your homework to the compass/blackboard system instead of turning in a hard copy. If your
solutions are hand written, points may be deducted if the handwriting is difficult to read.
You may bring two sheets of notes plus a copy of the supplementary notes to the first exam and three sheets of notes plus the supplementary notes to the second exam. You may use both sides of the sheets, with font size 10 or larger printing (or similar handwriting size). The examinations are closed book otherwise. Calculators, laptop computers, tables of integrals, etc. are not permitted.