Machine Learning Seminar

March 4th, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Suguman Bansal (COMP)

Please indicate interest, especially if you want lunch, here.

Abstract:

Intelligent machines, such as IoT devices, are fundamentally reactive systems that interact with the outer physical environment to achieve a goal.
Although prevalent, the design of a ‘provably correct’ intelligent machines would require to account for all the complex ways in which an environment and the machines may interact. This is challenging and error-prone. In this talk, I will introduce the paradigm of “Reactive synthesis” wherein the machine is automatically constructed from a high-level description of its goal. This way, an intelligent machine can be designed in a declarative fashion as opposed to an imperative fashion. This talk will cover recent theoretical and algorithmic advances in the area of reactive synthesis under the assumption of asynchrony and richer goal descriptions. It will also cover how you, an ML enthusiast/researcher, can contribute to this emerging field of research.

February 26th, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Tan Nguyen (ECE)

Please indicate interest, especially if you want lunch, here.

Abstract:

Stochastic gradient descent (SGD) with constant momentum and its variants such as Adam are the optimization algorithms of choice for training deep neural networks (DNNs). Since DNN training is incredibly computationally expensive, there is great interest in speeding up convergence. Nesterov accelerated gradient (NAG) improves the convergence rate of gradient descent (GD) for convex optimization using a specially designed momentum; however, it accumulates error when an inexact gradient is used (such as in SGD), slowing convergence at best and diverging at worst. In this paper, we propose Scheduled Restart SGD (SRSGD), a new NAG-style scheme for training DNNs. SRSGD replaces the constant momentum in SGD by the increasing momentum in NAG but stabilizes the iterations by resetting the momentum to zero according to a schedule. Using a variety of models and benchmarks for image classification, we demonstrate that, in training DNNs, SRSGD significantly improves convergence and generalization; for instance in training ResNet200 for ImageNet classification, SRSGD achieves an error rate of 20.93% vs. the benchmark of 22.13%. These improvements become more significant as the network grows deeper. Furthermore, on both CIFAR and ImageNet, SRSGD reaches similar or even better error rates with fewer training epochs compared to the SGD baseline.

February 19th, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Lisa O’Bryan (PSYC/ECE)

Please indicate interest, especially if you want lunch, here.

Abstract:

My research focuses on the critical role communication plays in the successful functioning of social groups, with attention to how communication patterns impact, and are impacted by, individual and group-level properties. I study this topic through the development of technology that can obtain detailed, time-varying measurements of individual behaviors and interactions. In this talk I will review my work using both tracking collars to study how vocalizations influence the collective movements of domesticated herds and wild baboons and 360 degree cameras to study the role conversational turn-taking plays in the decision-making and collective intelligence of human teams. Throughout the talk I will highlight the role machine learning plays in the processing of these high-resolution data. The long-term goal of my research program is to gain new insights into the evolution of communication systems and how we can engineer communication systems within our own societies to produce more favorable group-wide outcomes.

February 12th, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Tayo Ajayi (CAAM)

Please indicate interest, especially if you want lunch, here.

Abstract:

In radiation therapy treatment planning optimization, selecting a set of clinical objectives that are tractable and parsimonious yet clinically effective is a challenging task. In clinical practice, this is typically done by trial and error based on the treatment planner’s subjective assessment, which often makes the planning process inefficient and inconsistent. We develop the objective selection problem that infers a sparse set of objectives for prostate cancer treatment planning based on historical treatment data. We formulate the problem as a non-convex bilevel mixed-integer program using inverse optimization and highlight its connection with feature selection to propose greedy heuristics as well as application-specific methods that utilize anatomical information of the patients. Our results show that the proposed heuristics find objectives that are near optimal. Using curve analysis for dose-volume histograms, we show that the learned objectives closely represent latent clinical preferences by recovering historical treatment for each patient.

February 5, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Daniel McDuff and Mary Czerwinski (Microsoft Research)

Please indicate interest, especially if you want lunch, here.

Abstract:

Machine learning and AI present many opportunities for building more natural user interfaces. This presentation will cover examples of state-of-the-art human sensing and synthesis algorithms that can be used to create computer systems that adapt to users’ physiological states, expressions and style. It will address how these techniques can be used in end-to-end embodied conversational agents, what applications these might have in enhancing human productivity and well-being and give examples of challenges that exist in creating complex multimodal systems. Finally, these technologies present opportunities to create more intelligent machines; however, they also raise questions about the ethics of designing systems that measure and leverage highly personal data. The talk will contain proposals for design principals to help address these questions.

January 28, 12:00 pm – 1:00 pm in DCH 3092
Speaker: Ewa M. Nowara (ELEC)

Please indicate interest, especially if you want lunch, here.

Abstract:

Imagine if when you looked at someone, you could see their heartbeat. Using an off the shelf camera, we can do just that. As the blood flows through the skin, the varying blood concentration changes the color of the skin slightly over time. Analyzing these subtle color variations allows us to recover information related to vital signs, including heart rate, heart rate variability, and breathing rate, from a simple video recording of a skin region. Unfortunately, the skin color variations caused by flowing blood are extremely small and are easily corrupted by noise, so most existing methods only work in controlled settings in the lab.

I jointly design optical systems with machine learning and signal processing algorithms to understand and overcome the challenges associated with camera-based physiology measurements in realistic applications, including uncontrolled illumination present during driving, or high video compression during Skype teleconferencing. Imaging vital signs with cameras could enable new exciting applications not possible with the existing technology, e.g., Skype teleconferencing with physicians for people living far from a hospital or a smart car that could automatically pull over if it detects the driver had a cardiac arrest during driving.

January 22nd, 12:00 pm- 1:00 pm in DCH 3092
Speaker: Giuseppe Vinci

Please indicate interest, especially if you want lunch, here.
Abstract:

We investigate the problem of conditional dependence graph estimation when several pairs of nodes have no joint data observation. For these pairs even the simplest metric of covariability, the sample covariance, is unavailable. This problem arises, for instance, in calcium imaging recordings, where the activities of a large population of neurons are typically observed by recording from smaller subsets of cells at once, and several pairs of cells are never recorded simultaneously. With no additional assumption, the unavailability of parts of the covariance matrix translates into the unidentifiability of the precision matrix that, in the Gaussian graphical model setting, specifies the graph. Recovering a conditional dependence graph in such settings is an extremely hard challenge, because it requires to infer conditional dependences between network nodes with no empirical evidence of their covariability.

We call this challenge the “graph quilting problem”. We demonstrate that, under mild conditions, it is possible to correctly identify not only the edges connecting the observed pairs of nodes, but also a superset of those connecting the variables that are never observed jointly. We propose an L1-regularized graph estimator based on a partially observed sample covariance matrix and establish its rates of convergence in high-dimensions. We finally present a simulation study and apply the methods to calcium imaging data of ten thousand neurons in mouse visual cortex.

November 20th, 12:00 pm- 1:00 pm in DCH 3092
Speaker: Swarat Chaudhuri

Please indicate interest, especially if you want lunch, here.
Abstract:

Program synthesis, the problem of automatically discovering programs that implement a given specification, is a long-standing goal in computer science. The last decade has seen a flurry of activity in this area, mostly in the Programming Languages community. However, this problem is also of interest to Machine Learning researchers. This is because machine learning models, especially for complex, procedural tasks, are naturally represented as programs that use symbolic primitives for higher-level reasoning and neural networks for lower-level pattern recognition. Such “neurosymbolic”, programmatic models have a number of advantages: they can be easier to interpret and more amenable to automatic formal certification than purely neural models, permit the encoding of strong inductive biases, and facilitate the transfer of knowledge across learning settings. The learning of such models is a form of program synthesis.

In this talk, I will summarize some of our recent work on such program synthesis problems. In particular, I will describe PROPEL, a reinforcement learning framework in which policies are represented as programs in a domain-specific language, and learning amounts to synthesizing these programs. I will also describe HOUDINI, a learning framework that uses the modularity of a functional programming language to reuse neural modules across learning tasks. Collectively, these results point to a new way in which ideas from Programming Languages and Machine Learning can come together and help realize the dream of high-performance, reliable, and trustworthy intelligent systems.

November 13th, 12:00 pm- 1:00 pm in DCH 3092
Speaker: Vicky Yao (COMP)

Please indicate interest, especially if you want lunch, here.
Abstract:

The generation of diverse genome-scale data across organisms and experimental conditions is becoming increasingly commonplace, creating unprecedented opportunities fоr understanding the molecular underpinnings of human disease. However, realization of these opportunities relies on the development of novel computational approaches, as these large data are often noisy, highly heterogeneous, and lack the resolution required to study key aspects of metazoan complexity, such as tissue and cell-type specificity. Furthermore, targeted data collection and experimental verification is often infeasible in humans, underscoring the need for methods that can integrate -omics data, computational predictions, and biological knowledge across organisms. To address these challenges, I have developed diseaseQUEST, an integrative computational-experimental framework that combines human quantitative genetics with in silico functional network representations of model organism biology to systematically identify disease gene candidates. This framework leverages a novel semi-supervised Bayesian network integration approach to predict tissue- and cell-type specific functional relationships between genes in model organisms. We use diseaseQUEST to construct 203 tissue- and cell-type specific functional networks and predict candidate genes for 25 different human diseases and traits using C. elegans as a model system and focus on Parkinson’s disease as a case study. I will also talk about a related project that models the role of cell-type specificity in human disease, where I developed the first network models of Alzheimer’s-relevant neurons, in particular, the neuron type most vulnerable to the disease. We then use these models to predict and interpret processes that are critical to the pathological cascade of Alzheimer’s.

November 6th, 12:00 pm- 1:00 pm in DCH 3092
Speaker: Beidi Chen (COMP)

Please indicate interest, especially if you want lunch, here.
Abstract:

This talk will be in two parts:

1) The mechanisms behind human visual systems and convolutional neural networks (CNNs) are vastly different. Hence, it is expected that they have different notions of ambiguity or hardness. In this work, we make a surprising discovery: there exists a (nearly) universal score function for CNNs whose correlation with human visual hardness is statistically significant. We term this function as angular visual hardness (AVH) and in a CNN, it is given by the normalized angular distance between a feature embedding and the classifier weights of the corresponding target category. We conduct an in-depth scientific study. We observe that CNN models with the highest accuracy also have the best AVH scores. This agrees with an earlier finding that state-of-art models tend to improve on classification of harder training examples. We find that AVH displays interesting dynamics during training: it quickly reaches a plateau even though the training loss keeps improving. This suggests the need for designing better loss functions that can target harder examples more effectively. Finally, we empirically show significant improvement in performance by using AVH as a measure of hardness in self-training tasks.

2) Entity resolution identifies and removes duplicate entities in large, noisy databases and has grown in both usage and new developments as a result of increased data availability. Nevertheless, entity resolution has tradeoffs regarding assumptions of the data generation process, error rates, and computational scalability that make it a difficult task for real applications. In this work, we focus on a related problem of unique entity estimation, which is the task of estimating the unique number of entities and associated standard errors in a data set with duplicate entities. Unique entity estimation shares many fundamental challenges of entity resolution, namely, that the computational cost of all-to-all entity comparisons is intractable for large databases. To circumvent this computational barrier, we propose an efficient (near-linear time) estimation algorithm based on locality sensitive hashing. Our estimator, under realistic assumptions, is unbiased and has provably low variance compared to existing random sampling based approaches. In addition, we empirically show its superiority over the state-of-the-art estimators on three real applications. The motivation for our work is to derive an accurate estimate of the documented, identifiable deaths in the ongoing Syrian conflict. Our methodology, when applied to the Syrian data set, provides an estimate of 191, 874 ± 1772 documented, identifiable deaths, which is very close to the Human Rights Data Analysis Group (HRDAG) estimate of 191,369. Our work provides an example of challenges and efforts involved in solving a real, noisy challenging problem where modeling assumptions may not hold.