Resources on Parallel Computing
Machine learning systems are widely used in both industrial applications and academic research. This course will give an overview of the problems and solutions of high-performance machine learning systems. Presentations, coding, and written reports are required. The focus of the course is to demonstrate the implementation mechanisms underneath popular machine learning tools, e.g., tensowflow, pytorch, xgboost. Students will implement high-performance machine learning systems from scratch. This special-topics instance is classified as belonging to the Data Management and the Intelligent Systems clusters.
- CSCI-335 or CSCI-635 or
- Equivalent or permission of instructor
The students are expected to be familiar with Python, a parallel version of Python, MPI, C/C++, and OpenMP. In case not, the students are expected to learn the basics themselves. If you plan on using other languages or framework, be sure to check with the instructor.
- Support vector machines, logistic regression, and gradient descent. Parallel implementations of gradient descent (and its variants) to optimize support vector machines and logistic regressions. The parallel implementation covers how to efficiently utilize multi-threading and GPUs to accelerate the computation. A brief introduction to multi-threading and GPU programming basics is included. (4 weeks)
- Gradient boosting decision trees. Parallel split, quantization implementations, and optimizations on GPUs for tree models are discussed. (3 weeks)
- Neural network preliminary. Parallel tensor operator implementations. Lazy evaluation and code generation. (2 weeks)
- Neural network basic layers. Parallel implementations for backpropagation. (2 weeks)
- Approximate nearest neighbor searching framework. Parallel implementation for locality sensitive hashing, quantization, and proximity graph on CPUs and GPUs. (3 weeks)
I will use the following percentage-based grading scale to determine your
final letter grade.
Each in-class activity will be due on a specified date, and be graded like
homework assignment. NO LATE submission will be accepted.
NO DEADLINE EXTENSION will be given unless extenuating circumstances arise.
Any student who violates the academic integrity standards will fail the course (even for the first offense).
Any missed exams, projects, or homework assignments will get zero.
Class attendance is very important.
If you miss a class, it is your responsibility to acquire any missed
Note that questions about grading must be brought to my attention within
one week after the graded material has been handed back. After that time,
the grade will become permanent.
No extra work will be given to help students to raise grades due to fairness
I expect students to respect their instructor, teaching assistants, and
other students in class. Disrespectful behavior will NOT be tolerated.
Do NOT come to class late, leave early, or talk to other students, etc.
However, class participation and discussion are strongly encouraged.
Any form of academic dishonesty is strictly prohibited. I will handle any such
incident according to the Department of Computer Science
Policy on Academic
Dishonesty. Violations of the
Code of Conduct for the Use of Department of Computer Science Facilities
can also result in suspension, expulsion and even criminal charges.
Please refer to the following statements excerpted from the
1998-99 RIT Students Rights & Responsibilities handbook:
Any act of improperly representing another person's work as one's own is construed as an act of academic dishonesty. These acts include, but are not limited to, plagiarism in any form, or use of information and materials not authorized by the instructor during an examination.
If a faculty member judges a student to be guilty of some form of academic
dishonesty, the student may be given a failing grade for that piece of work or for the course, depending upon the severity of the misconduct.
For the record, I have adopted the following standard policy on academic honesty.
You may discuss general issues with other students, or use other reference materials. Here, the general discussion precludes the detailed techniques or algorithms for solutions. In this case, any help you receive from someone or the references must be acknowledged. Failure to acknowledge the source of a significant idea or approach will be considered cheating or plagiarism.
Note that this does not mean that someone else can do your work.
Unless otherwise explicitly specified, all programming projects and written assignments you submit must be done independently. You are not allowed to look at another student's code/solutions or use the code that others have submitted in the past.
The corollary is that you may not do someone else's work for them either. A willing supplier of the material is as guilty of academic dishonesty as the receiver. Students are responsible for securing their work -- printed copies are not left in public places, and file/directory permissions are set to be unreadable.
Policy on W and I Grades
RIT policy allows you to withdraw from a course with a grade of W
on or before the Friday of the sixth week in the quarter. After this date,
your instructor cannot give you a W, but must assign you a grade based on your work.
This course has been designed so that you can complete all the work in one semester. Thus incomplete grades will be given only in the most exceptional circumstances, and then only by prior arrangement with the instructor who has the final say in this matter.
Policy Prohibiting Discrimination and Harrasment
RIT is committed to providing a safe learning environment, free of harassment and discrimination as articulated in our university policies located on our governance website. RIT's policies require faculty to share information about incidents of gender based discrimination and harassment with RIT's Title IX coordinator or deputy coordinators, regardless whether the incidents are stated to them in person or shared by students as part of their coursework.
If you have a concern related to gender-based discrimination and/or harassment and prefer to have a confidential discussion, assistance is available from one of RIT's confidential resources on campus (listed below).
- The Center for Women & Gender: Campus Center Room 1760; 585-475-7464; CARES (available 24 hours/7 days a week) Call or text 585-295-3533.
- RIT Student Health Center--August Health Center/1st floor; 585-475-2255.
- RIT Counseling Center--August Health Center /2nd floor - 2100; 585-475-2261.
- The Ombuds Office--Student Auxiliary Union/Room 1114; 585-475-7200 or 585-475-2876.
- The Center for Religious Life--Schmitt Interfaith Center/Rm1400; 585-475-2137.
- NTID Counseling & Academic Advising Services--2nd Floor Lynden B. Johnson; 585-475-6468 (v), 585-286-4070 (vp).
Any part of this page is subject to change any time. In that case, you will be informed in advance. Be alert to the announcements.
Last updated 04/30/22