I will present three recent projects within the 3D Deep Learning research line from my team at Google Research: (1) a deep network for reconstructing the 3D shape of multiple objects appearing in a single RGB image (ECCV'20). (2) a new conditioning scheme for normalizing flow models. It enables several applications such as reconstructing an object's 3D point cloud from an image, or the converse problem of rendering an image given a 3D point cloud, both within the same modeling framework (CVPR'20); (3) a neural rendering framework that maps a voxelized object into a high quality image. It renders highly-textured objects and illumination effects such as reflections and shadows realistically. It allows controllable rendering: geometric and appearance modifications in the input are accurately represented in the final rendering (CVPR'20).
Game Development requires a vast array of tools, techniques, and expertise, ranging from game design, artistic content creation, to data management and low level engine programming. Yet all of these domains have one kind of task in common - the transformation of one kind of data into another. Meanwhile, advances in Machine Learning have resulted in a fundamental change in how we think about these kinds of data transformations - allowing for accurate and scalable function approximation, and the ability to train such approximations on virtually unlimited amounts of data. In this talk I will present how these two fundamental changes in Computer Science affect game development - how they can be used to improve game technology as well as the way games are built - and the exciting new possibilities and challenges they bring along the way.
Organizers: Abhinanda Ranjit Punnakkal
The precise delivery of bio-functionalized matters is of great interest from the fundamental and applied viewpoints. Particularly, most existing single cell platforms are unable to achieve large scale operation with flexibility on cells and digital manipulation towards multiplex cell tweezers. Thus, there is an urgent need of innovative techniques to accomplish the automation of single cells. Recently, the flexibility of magnetic shuttling technology using nano/micro scale magnets for the manipulation of particles has gained significant advances and has been used for a wide variety of single cells manipulation tasks. Herein, let’s call “spintrophoresis” using micro-/nano-sized Spintronic devices rather than “magnetophoresis” using bulk magnet. Although a digital manipulation of single cells has been implemented by the integrated circuits of spintrophoretic patterns with current, active and passive sorting gates are required for its practical application for cell analysis. Firstly, a universal micromagnet junction for passive self-navigating gates of microrobotic carriers to deliver the cells to specific sites using a remote magnetic field is described for passive cell sorting. In the proposed concept, the nonmagnetic gap between the defined donor and acceptor micromagnets creates a crucial energy barrier to restrict particle gating. It is shown that by carefully designing the geometry of the junctions, it becomes possible to deliver multiple protein- functionalized carriers in high resolution, as well as MFC-7 and THP-1 cells from the mixture, with high fidelity and trap them in individual apartments. Secondly, a convenient approach using multifarious transit gates is proposed for active sorting of specific cells that can pass through the local energy barriers by a time-dependent pulsed magnetic field instead of multiple current wires. The multifarious transit gates including return, delay, and resistance linear gates, as well as dividing, reversed, and rectifying T-junction gates, are investigated theoretically and experimentally for the programmable manipulation of microrobotic particles. The results demonstrate that, a suitable angle of the 3D-gating field at a suitable time zone is crucial to implement digital operations at integrated multifarious transit gates along bifurcation paths to trap microrobotic carriers in specific apartments, paving the way for flexible on-chip arrays of multiplexed cells. Finally, I will include the pseudo-diamagnetic spintrophoresis using negative magnetic patterns for multiplexed magnetic tweezers without the biomarker labelling. Label free single cells manipulation, separation and localization enables a novel platform to address biologically relevant problems in bio-MEMS/ NEMS technologies.
Biological motion is fascinating in almost every aspect you look upon it. Especially locomotion plays a crucial part in the evolution of life. Structures, like the bones connected by joints, soft and connective tissues and contracting proteins in a muscle-tendon unit enable and prescribe the respective species' specific locomotion pattern. Most importantly, biological motion is autonomously learned, it is untethered as there is no external energy supply and typical for vertebrates, it's muscle-driven. This talk is focused on human motion. Digital models and biologically inspired robots are presented, built for a better understanding of biology’s complexity. Modeling musculoskeletal systems reveals that the mapping from muscle stimulations to movement dynamics is highly nonlinear and complex, which makes it difficult to control those systems with classical techniques. However, experiments on a simulated musculoskeletal model of a human arm and leg and real biomimetic muscle-driven robots show that it is possible to learn an accurate controller despite high redundancy and nonlinearity, while retaining sample efficiency. More examples on active muscle-driven motion will be given.
Organizers: Ahmed Osman
Feedback based automatic control has been a key enabling technology for many technological advances over the past 80 years. New application domains, like autonomous cars driving on automated highways, energy distribution via smart grids, life in smart cities or the new production paradigm Industry 4.0 do, however, require a new type of cybernetic systems and control theory that goes beyond some of the classical ideas. Starting from the concept of feedback and its significance in nature and technology, we will present in this talk some new developments and challenges in connection to the control of today's and tomorrow’s intelligent systems.
Clearly explaining a rationale for a classification decision to an end-user can be as important as the decision itself. Existing approaches for deep visual recognition are generally opaque and do not output any justification text; contemporary vision-language models can describe image content but fail to take into account class-discriminative image properties which justify visual predictions. In this talk, I will present my past and current work on Zero-Shot Learning, Vision and Language for Generative Modeling and Explainable Machine Learning where we show (1) how to generalize image classification models to cases when no labelled visual training data is available, (2) how to generate images and image features using detailed visual descriptions, and (3) how our models focus on discriminating properties of the visible object, jointly predict a class label, explain why/not the predicted label is chosen for the image.
During manipulation, humans adjust the amount of force applied to an object depending on friction: they exert a stronger grip for slippery surfaces and a looser grip for sticky surfaces. However, the neural mechanisms signaling friction remain unclear. To fill this gap, we recorded the response of human tactile afferent during the onset of slip against flat surfaces of different frictions. We observed that some afferents responded to partial slip events occurring during transition from a stuck to a slipping contact, and potentially signaling the impending slip.
Wearable sensing and feedback devices are becoming increasingly ubiquitous for measuring human movement in research laboratories, medical clinics, and in consumer goods. Advances in computation and miniaturization have enabled sensing for gait assessment; these technologies are then used in interventions to provide feedback that facilitates changes in gait or enhances sensory capabilities. This talk will focus on vibration as the primary method of providing feedback. I will discuss the use of vibrotactile arrays to communicate plantar foot pressure in users of lower-limb prosthetics, as a synthetic form of sensory feedback. Wearable vibrating units can also be used as a cue to retrain gait, and I will describe my preliminary work in gait retraining as a conservative treatment for knee osteoarthritis. This talk will cover the development and evaluation of these haptic devices and establish their impact within the greater context of clinical biomechanics.
Search-based Planning refers to planning by constructing a graph from systematic discretization of the state- and action-space of a robot and then employing a heuristic search to find an optimal path from the start to the goal vertex in this graph. This paradigm works well for low-dimensional robotic systems such as mobile robots and provides rigorous guarantees on solution quality. However, when it comes to planning for higher-dimensional robotic systems such as mobile manipulators, humanoids and ground and aerial vehicles navigating at high-speed, Search-based Planning has been typically thought of as infeasible. In this talk, I will describe some of the research that my group has done into changing this thinking. In particular, I will focus on two different principles. First, constructing multiple lower-dimensional abstractions of robotic systems, solutions to which can effectively guide the overall planning process using Multi-Heuristic A*, an algorithm recently developed by my group. Second, using offline pre-processing to provide a *provably* constant-time online planning for repetitive planning tasks. I will present algorithmic frameworks that utilize these principles, describe their theoretical properties, and demonstrate their applications to a wide range of physical high-dimensional robotic systems.
Security and privacy is of growing concern in many control applications. Cyber attacks are frequently reported for a variety of industrial and infrastructure systems. For more than a decade the control community has developed techniques for how to design control systems resilient to cyber-physical attacks. In this talk, we will review some of these results. In particular, as cyber and physical components of networked control systems are tightly interconnected, it is be argued that traditional IT security focusing only on the cyber part does not provide appropriate solutions. Modeling the objectives and resources of the adversary together with the plant and control dynamics is shown to be essential. The consequences of common attack scenarios, such denial-of-service, replay, and bias injection attacks, can be analyzed using the presented framework. It is also shown how to strengthen the control loops by deriving security and privacy aware estimation and control schemes. Applications in building automation, power networks, and automotive systems will be used to motivate and illustrate the results. The presentation is based on joint work with several students and colleagues at KTH and elsewhere.
In the search for materials with new properties, there have been great advances in recent years aimed at the construction of mechanical systems whose behaviour is governed by structure, rather than composition. Through careful design of the material’s architecture, new material properties have been demonstrated, including negative Poisson’s ratio, high stiffness-to-weight ratio and mechanical cloaking. While originally the field focused on achieving unusual (zero or negative) values for familiar mechanical parameters, more recently it has been shown that non-linearities can be exploited to further extend the design space. In this talk Prof. Katia Bertoldi will focus on kirigami-inspired metamaterials, which are produced by introducing arrays of cuts into thin sheets. First, she will demonstrate that instabilities triggered under uniaxial tension can be exploited to create complex 3D patterns and even to guide the formation of permanent folds. Second, she will show that such non-linear systems can be used to designs smart and flexible skins with anisotropic frictional properties that enables a single soft actuator to propel itself. Finally, Prof.Bertoldi will focus on bistable kirigami metamaterials and show that they provide an ideal environment for the propagation non-linear waves.
Organizers: Metin Sitti
Machine learning increasingly supports consequential decisions in domains including health, employment, and criminal justice. Consequential decision making is inherently dynamic: Individuals, their outcomes, and entire populations can change and adapt in response to classification. Traditional machine learning, however, fails to account for such dynamic effects. In this talk, I will highlight three different vignettes of dynamic decision making. The first is about how classification changes populations and how this perspective is essential to questions of fairness in machine learning. The second is about how classification incentivizes individuals to adapt strategically. The third is about how predictions are often performative, that is, they influence the very outcome they aim to predict. I will end on the contours of a theory that unifies these three settings and its connections to questions in causality, control theory, economics, and sociology.
Organizers: Metin Sitti