CAP 5636 - Advanced Artificial Intelligence

Fall 2020

Class description: Principles of artificial intelligence. Uninformed and informed search. Constraint satisfaction. AI for game playing. Probabilistic reasoning, Markov decision processes, hidden Markov models, Bayes nets. Neural networks and deep learning.
Course objectives: By the end of the semester the students will be able to:
  • understand the search and decision making techniques used in modern artificial intelligence
  • apply artificial intelligence techniques in their own code
  • understand the societal and ethical implications of artificial intelligence
Instructor: Dr. Lotzi Bölöni
Office: HEC - 319
Phone: (407) 823-2320 (on last resort)
E-mail: (preferred means of communication)
TA: Mohamed Elfeki
Web Site:
The assignments and the other announcements will be posted on the course web site
Classroom: online
Class Hours: Tue, Th 12:00PM - 1:15PM
Office Hours: Mon, Wed 6:00PM - 7:30PM
See webcourse announcement for Zoom link.
Pre-requisites: CAP 4630, or consent of instructor.
Required texts: There is no required textbook.
Recommended readings:
  • Stuart Russel and Peter Norvig, Artificial Intelligence - A Modern Approach, 4rd edition
  • Only full grades will be used based on the points obtained. A for 90 and above, B for 80-89, C for 70-79, F for lower than 70.
  • Points awarded: Midterm 1: 20 points, Midterm 2: 20 points, Homeworks/Projects: 30 points total, Final exam 30 points.
  • Some midterms, exams and homeworks will have bonus points, but no curve will be applied.
  • The exams will be administered through ProctorHub, and are open book, open notes.
  • Make up exams will be given only in justified cases.
Sample exams Sample Midterm 1
Sample Midterm 2
Sample Final Exam
Note: you should not expect that the new exams are just variations with different data.
Integrity: The department, college, and University are committed to honesty and integrity in all academic matters. We do not tolerate academic misconduct by students in any form, including cheating, plagiarism and commercial use of academic materials. Please consult the Golden Rule Handbook for the procedures which will be applied.
Verification of engagement: As of Fall 2014, all faculty members are required to document students' academic activity at the beginning of each course. In order to document that you began this course, please complete the following academic activity by the end of the first week of classes, or as soon as possible after adding the course, but no later than August 27. Failure to do so will result in a delay in the disbursement of your financial aid.
To satisfy this requirement, you must finish the first quiz posted online. Log in to Webcourses, choose CAP 5636, and submit your answers online.
Course accessibility: The University of Central Florida is committed to providing access and inclusion for all persons with disabilities. Students should connect with Student Accessibility Services (Ferrell Commons 185,, phone (407) 823-2371). Through Student Accessibility Services, a Course Accessibility Letter may be created and sent to professors, which informs faculty of potential access and accommodations that might be reasonable. If you are a deployed active duty military student and feel that you may need a special accommodation due to that unique status, please contact your instructor to discuss your circumstances.
Campus safety: Emergencies on campus are rare, but if one should arise in our class, everyone needs to work together. Students should be aware of the surroundings and familiar with some basic safety and security concepts.
  • In case of an emergency, dial 911 for assistance.
  • Every UCF classroom contains an emergency procedure guide posted on a wall near the door. Please make a note of the guide's physical location and consider reviewing the online version.
  • If there is a medical emergency during class, we may need to access a first aid kit or AED (Automated External Defibrillator). To learn where those items are located in this building, see the link (click on link from menu on left).
  • To stay informed about emergency situations, sign up to receive UCF text alerts by going to and logging in. Click on "Student Self Service" located on the left side of the screen in the tool bar, scroll down to the blue "Personal Information" heading on your Student Center screen, click on "UCF Alert," fill out the information, including your e-mail address, cell phone number, and cell phone provider, click "Apply" to save the changes, and then click "OK."
Zoom for Remote Instruction Because of the continued remote instruction requirement due to the COVID-19 pandemic, this course will use Zoom for synchronous ("real time") class meetings.

Please take the time to familiarize yourself with Zoom by visiting the UCF Zoom Guides at . You may choose to use Zoom on your mobile device (phone or tablet).

Things to Know About Zoom:

  • You must sign in to my Zoom session using your UCF NID and password.
  • The Zoom sessions are recorded.
  • Improper classroom behavior is not tolerated within Zoom sessions and may result in a referral to the Office of Student Conduct.
  • You can contact Webcourses@UCF Support at if you have any technical issues accessing Zoom.

Please see the Zoom housekeeping slide developed by UCF instructional designer, Trudy Trail-Constant.

COVID-19 Statements University-Wide Face Covering Policy for Common Spaces and Face-to-Face Classes

To protect members of our community, everyone is required to wear a facial covering inside all common spaces including classrooms ( Students who choose not to wear facial coverings will be asked to leave the classroom by the instructor. If they refuse to leave the classroom or put on a facial covering, they may be considered disruptive (please see the Golden Rule for student behavior expectations). Faculty have the right to cancel class if the safety and well-being of class members are in jeopardy. Students will be responsible for the material that would have been covered in class as provided by the instructor.

Notifications in Case of Changes to Course Modality

Depending on the course of the pandemic during the semester, the university may make changes to the way classes are offered. If that happens, please look for announcements or messages in Webcourses@UCF or Knights email about changes specific to this course.

COVID-19 and Illness Notification

Students who believe they may have a COVID-19 diagnosis should contact UCF Student Health Services (407-823-2509) so proper contact tracing procedures can take place.

Students should not come to campus if they are ill, are experiencing any symptoms of COVID-19, have tested positive for COVID, or if anyone living in their residence has tested positive or is sick with COVID-19 symptoms. CDC guidance for COVID-19 symptoms is located here: (

Students should contact their instructor(s) as soon as possible if they miss class for any illness reason to discuss reasonable adjustments that might need to be made. When possible, students should contact their instructor(s) before missing class.

In Case of Faculty Illness

Depending on the course of the pandemic during the semester, the university may make changes to the way classes are offered. If that happens, please look for announcements or messages in Webcourses@UCF or Knights email about changes specific to this course.

Course Accessibility and Disability COVID-19 Supplemental Statement

Accommodations may need to be added or adjusted should this course shift from an on-campus to a remote format. Students with disabilities should speak with their instructor and should contact to discuss specific accommodations for this or other courses.


Lecture Notes, Readings, Homeworks
Tue, Aug. 25
History and positioning of AI
  • Motivating AI. Dangers of AI and AGI.
  • Early history
  • Expert systems
[slides] History and positioning of AI
Thu, Aug. 27
History and positioning of AI
  • Neural networks
  • The two intellectual traditions of logic vs neural networks
  • A melting pot of other ideas
  • The agent view of AI

Tue, Sep. 1
Uninformed search
  • Reflex agents
  • Search problems
  • Depth first and breadth first search
  • Uniform cost search

[slides] Uninformed search
Thu, Sep. 3

Tue, Sept. 8

Thu, Sep. 10
Informed search: A* search and heuristics
  • Informed search methods
  • Heuristics
  • Greedy search
  • A* search
  • Graph search
[slides] Informed search
Tue, Sep. 15
Game playing and adversarial search
  • Types of games
  • Adversarial search, minimax
  • The problem of depth
  • Evaluation functions
  • Alpha Beta pruning
[slides] Adversarial search
Thu, Sep. 17 Expectimax search and utilities
  • Expectimax search
  • Refresher about probabilities
  • Utilities and rationality
[slides] Expectimax search and utilities
Tue, Sep. 22
Markov decision processes 1
  • Defining MDPs: policies and utilities
  • Optimal policy, value of state, value of Q-state
[slides] Markov Decision Processes 1
Thu, Sept. 24
Midterm 1 - Introduction to Expectimax
Tue, Sep. 29
Markov decision processes 2
  • Policy iteration
[slides] Markov Decision Processes 2
Thu, Oct. 1
Reinforcement learning 1
  • Reinforcement learning as a twist on MDPs
[slides] Reinforcement learning 1
Tue, Oct. 6
  • Model-based and model-free learning
  • Temporal difference learning

Thu, Oct. 8
Reinforcement learning 2
  • Exploration vs. exploitation, regret
  • Generalization across states
  • Policy search
[slides] Reinforcement learning 2
Tue, Oct. 13
  • Random variables
  • Joint and marginal distributions, conditional distribution
[slides] Probability
Thu, Oct. 15
  • Product rule, chain rule, Bayes' rule
  • Inference
  • Independence

Tue, Oct. 20
Markov models
  • Markov chains
  • Conditional independence
  • Stationary distributions
[slides] Markov models
Thu, Oct. 22
Hidden Markov models
  • Hidden Markov models
  • Example: robot localization
Homework 2 - Reinforcement learning - due November 15, 2020
[slides] Hidden Markov models
Tue, Oct. 27
  • Most likely explanation
  • Speech recognition
Th, Oct. 29
Tue, Nov. 3
Midterm 2 - from MDP to Markov Chains
Thu, Nov. 5
Particle filters and applications of HMMs
  • Particle filters
  • Robot localization with particle filters
  • Dynamic Bayes nets
[slides] Particle filters and Applications of HMMs
Tue, Nov. 10
Classification, principles of machine learning, naive Bayes
  • Classification
  • Model-based classification
  • Naive Bayes
  • Spam filter example
  • Generalization and overfitting
  • Parameter estimation

[slides] Classification and naive Bayes
Thu, Nov. 12
  • Classification and machine learning cont'd
Tue, Nov. 17
Introduction to deep learning
  • History and impact

[slides] DL1 - Introduction
Thu, Nov. 19
Machine learning background of deep learning
  • History and impact
  • Machine learning background
  • Loss functions: squared, cross-entropy, softmax
  • Optimization, stochastic gradient descent
  • Backpropagation
[slides] DL2 - Machine learning background
Homework 3 - due December 3

Tue, Nov. 24
Feedforward neural networks
  • Feedforward networks
  • Stochastic gradient descent
[slides] DL3 - Feedforward neural networks
Thu, Nov. 26
Thanksgiving break - no class

Tue, Dec. 1
Convolutional neural networks
  • Convolutions
  • Convolutional filters in neural networks
  • Pooling layers
[slides] DL4 - Convolutional neural networks
Thu, Dec. 3
Future of AI
  • General definitions of intelligence
  • Cognitive architectures
  • Superintelligence and singularity

Thu, Dec. 10

Final exam Thursday December 10, 10:00 AM - 12:50 PM