Topics in Intelligent Systems---Machine Learning Systems Implementation
CSCI-739

Course Overview

Machine learning systems are widely used in both industrial applications and academic research. This course will give an overview of the problems and solutions of high-performance machine learning systems. Presentations, coding, and written reports are required. The focus of the course is to demonstrate the implementation mechanisms underneath popular machine learning tools, e.g., tensowflow, pytorch, xgboost. Students will implement high-performance machine learning systems from scratch. This special-topics instance is classified as belonging to the Data Management and the Intelligent Systems clusters.

Prerequisites

• CSCI-335 or CSCI-635 or
• Equivalent or permission of instructor

Expectations

The students are expected to be familiar with Python, a parallel version of Python, MPI, C/C++, and OpenMP. In case not, the students are expected to learn the basics themselves. If you plan on using other languages or framework, be sure to check with the instructor.

Tentative Schedule

• Support vector machines, logistic regression, and gradient descent. Parallel implementations of gradient descent (and its variants) to optimize support vector machines and logistic regressions. The parallel implementation covers how to efficiently utilize multi-threading and GPUs to accelerate the computation. A brief introduction to multi-threading and GPU programming basics is included. (4 weeks)
• Gradient boosting decision trees. Parallel split, quantization implementations, and optimizations on GPUs for tree models are discussed. (3 weeks)
• Neural network preliminary. Parallel tensor operator implementations. Lazy evaluation and code generation. (2 weeks)
• Neural network basic layers. Parallel implementations for backpropagation. (2 weeks)
• Approximate nearest neighbor searching framework. Parallel implementation for locality sensitive hashing, quantization, and proximity graph on CPUs and GPUs. (3 weeks)

Grading

In-Class Activities

General Policies

Academic Honesty

Any form of academic dishonesty is strictly prohibited. I will handle any such incident according to the Department of Computer Science Policy on Academic Dishonesty. Violations of the Code of Conduct for the Use of Department of Computer Science Facilities can also result in suspension, expulsion and even criminal charges. Please refer to the following statements excerpted from the 1998-99 RIT Students Rights & Responsibilities handbook:

Any act of improperly representing another person's work as one's own is construed as an act of academic dishonesty. These acts include, but are not limited to, plagiarism in any form, or use of information and materials not authorized by the instructor during an examination.

If a faculty member judges a student to be guilty of some form of academic dishonesty, the student may be given a failing grade for that piece of work or for the course, depending upon the severity of the misconduct.

For the record, I have adopted the following standard policy on academic honesty.

• You may discuss general issues with other students, or use other reference materials. Here, the general discussion precludes the detailed techniques or algorithms for solutions. In this case, any help you receive from someone or the references must be acknowledged. Failure to acknowledge the source of a significant idea or approach will be considered cheating or plagiarism.
• Note that this does not mean that someone else can do your work. Unless otherwise explicitly specified, all programming projects and written assignments you submit must be done independently. You are not allowed to look at another student's code/solutions or use the code that others have submitted in the past.
• The corollary is that you may not do someone else's work for them either. A willing supplier of the material is as guilty of academic dishonesty as the receiver. Students are responsible for securing their work -- printed copies are not left in public places, and file/directory permissions are set to be unreadable.

Policy on W and I Grades

RIT policy allows you to withdraw from a course with a grade of W on or before the Friday of the sixth week in the quarter. After this date, your instructor cannot give you a W, but must assign you a grade based on your work.

This course has been designed so that you can complete all the work in one semester. Thus incomplete grades will be given only in the most exceptional circumstances, and then only by prior arrangement with the instructor who has the final say in this matter.

Policy Prohibiting Discrimination and Harrasment

RIT is committed to providing a safe learning environment, free of harassment and discrimination as articulated in our university policies located on our governance website. RIT's policies require faculty to share information about incidents of gender based discrimination and harassment with RIT's Title IX coordinator or deputy coordinators, regardless whether the incidents are stated to them in person or shared by students as part of their coursework. If you have a concern related to gender-based discrimination and/or harassment and prefer to have a confidential discussion, assistance is available from one of RIT's confidential resources on campus (listed below).

• The Center for Women & Gender: Campus Center Room 1760; 585-475-7464; CARES (available 24 hours/7 days a week) Call or text 585-295-3533.
• RIT Student Health Center--August Health Center/1st floor; 585-475-2255.
• RIT Counseling Center--August Health Center /2nd floor - 2100; 585-475-2261.
• The Ombuds Office--Student Auxiliary Union/Room 1114; 585-475-7200 or 585-475-2876.
• The Center for Religious Life--Schmitt Interfaith Center/Rm1400; 585-475-2137.
• NTID Counseling & Academic Advising Services--2nd Floor Lynden B. Johnson; 585-475-6468 (v), 585-286-4070 (vp).

Notes

Any part of this page is subject to change any time. In that case, you will be informed in advance. Be alert to the announcements.