Cyber-physical systems (CPSs) are inseparable parts of our future society that attract more interest from scholars and the young generation every day. Many dynamic systems including power and energy systems, autonomous cars, and connected vehicles in future transportations fall into the CPS category facing many scientific and technological challenges. There is a need for tools and methods for the secure, efficient, and reliable operation of such systems. The goal of my research is to integrate control theoretics with machine learning tools to enhance the resiliency, efficiency, and reliability of CPSs with applications to power systems, clean energy, and intelligent transportation systems.
Theory: Control, optimization, estimation, diagnostics of systems modeled via partial differential equations (PDE)/ ordinary differential equations (ODE), reinforcement learning, machine learning.
Research Area 1: Security and Resilient Control of Distributed Cyber Physical Systems
The complexity of modern engineering systems has enhanced attention toward the integration and coordination of communication networks, control systems, and physical plants. These high-dimensional systems, known as CPSs, operate in fundamental infrastructures e.g., power systems, transportations, energy, and manufacturing. In safety-critical applications of CPSs, failures can cause irreparable harm to the physical system as well as to humans interacting with CPS. Note that CPSs are not only subjected to physical failures, but they are also vulnerable to cyber-attacks and communication network failures. The effective operation of such critical infrastructures depends on secure and health-conscious control. Therefore, real-time resilient control strategies play a pivotal role in the reliable performance of such infrastructures despite potential cyber-attacks and network failures.
Goals of this project : Designing such controllers is a challenging problem due to the high dimensionality of CPSs, lack of global information, system constraints, and stability requirements. As a part of my recent projects, I have developed a distributed resilient algorithm based on hybrid control, machine learning techniques, and robust methodologies. The resilient algorithm maintains the performance of the CPS close to normal even in the presence of physical faults, network failures, or cyber-attacks.
Research Area 2 : Multi-objective Optimization for Resilient Smart Cities – Renewable integration with power systems
This research designs a (near) real-time framework to address interdisciplinary challenges of the control strategy design for large-scale multi-disciplinary systems e.g., smart cities. Figure below represents an overview of my research on energy management of smart cities as an example of a large-scale dynamical system with renewables integration including PV, large-scale batteries, and electrical vehicles (EVs). With higher penetration levels of renewables in power systems, the stability, and reliability of smart grids as well as energy market price require new techniques to overcome various challenges.
For the reliable performance of a large-scale interconnected system, I developed a novel approach that (i) provides a low computational effort algorithm for monitoring and estimation of system requirements for the near future (in our case study of smart grids, this module predicts day-ahead load requirements of the power system). The prediction model is designed based on a new ensemble learning algorithm that analyzes extracted essential features for predication; (ii) updates policies among multiple stakeholders (in case of smart grids, the policies related to updating the energy market price, considering benefits for renewables, Intelligent Charging Stations (ICSs), and EVs; and (iii) design low-level controllers as local controllers for distributed sectors (in case of smart grid, the controller are designed for utilities, microgrid’s center, EV owners, and ICSs) to satisfy all participants’ interests via cooperative game theory techniques. This research area develops a solution that goes beyond one specific service and extends multi-disciplinary multi-objective systems capabilities for various scenarios. This research has been awarded by NSF (PI, grant no. 2153438). We have developed a hardware-in-the-loop (HIL) system with physical components integrated into the OPAL-RT power system simulator to evaluate our proposed methods. This experimental setup is useful for various applications of control on dynamic interconnected systems.
Research Area 3 : Machine learning/Artificial Intelligence for Autonomous Systems
3.1: A Multi-Level Cyber-Survivability and Anti-Fragility Framework with an Application to Safety-Critical Embedded Systems
Objectives: Most of the existing resiliency algorithms are less competent for both effectiveness and real-time capability requirements simultaneously. This project goes beyond existing detection and mitigation methods and focuses on attack resiliency and learning from the experience to develop an anti-fragility mechanism with a low computational cost, high accuracy. rate, high capability of learning, and reliability for safety-critical embedded systems. Our framework identifies anomalies in (near) real time manner and develop resiliency measurements to guarantee the normal performance despite cyber threats.
3.2: Cyber Agility against Multi-Pronged Attacks: Architecture, Detection, and Mitigation
Objectives: This research will develop methods, algorithms, models, and hardware/software computing systems for harnessing fast-emerging techniques in machine learning with analytical approaches to provide the necessary means for detection, mitigation, and recovery from threats on safety-critical embedded systems. For AI-enabled critical embedded systems, cyber disruption via hardware/software attacks can impact many of the systems’ responsibilities. For such systems, mere monitoring and detection is not sufficient, and a full end-to-end framework including prevention, detection, and agile mitigation to maintain system performance is essential.
DISTRIBUTION STATEMENT A. Approved for public release Sept. 9, 2024: distribution unlimited. (AFRL-2024-4940)
Research Area 4 : Safe Reinforcement Learning using Control Lyapunov and Barrier methods
Safety critical applications involving expensive systems and human interaction require formal guarantees in terms of satisfying safety constraints during not only testing/deployment but training as well. Safe reinforcement learning (RL) aims to provide safety guarantees in terms of constraint satisfaction and stabilizing control policy. Policy derived through RL (model-based or model-free) do not incorporate constraints or provide theoretical guarantees for a stabilizing policy, furthermore, theoretical guarantees on system state constraints in most dynamical systems are not formally incorporated and guaranteed. Most real-world applications require some form of theoretical guarantees for safety.
Lyapunov functions provide theoretical guarantees for stability about a fixed point. Barrier functions provide theoretical guarantees for forward invariance of a set (interpreted as a constraint on the system). Forward invariance implies trajectories of states staying in a set for all time if it enters the set. Incorporation of these with RL can provide theoretical guarantees in terms of constraint satisfaction and stability.
Goal of this project: The objective of this research is to explore implicit and explicit methods to incorporate control Lyapunov and Barrier methods for Safe RL to further explore rates of convergence, constraint violation and effect on exploration during learning.
Research Area 5 : Load control in microgrids based on game theoretic methods
The transition to Electric Vehicles (EVs) has attracted more attention recently due to the increased energy efficiency and reduced CO2 emissions compared to conventional vehicles. The impact of EV integration can be both negative and positive. Excessive integration of EVs into the distribution network can impact the load profile, distribution system component capacity, voltage and frequency imbalances, excessive harmonic injection, power losses and the stability of the distribution grid. However, at the same time, EVs use batteries as their primary power sources which can also provide reserve service in Vehicle-to-Grid (V2G) systems to realize power management, power quality improvement, voltage and frequency regulation and renewable energy integration support. Considering different interests of users in the microgrid may conflict with each other, the game-theoretic method is a good analytical tool to address the energy trading problem.
5.1 : Bargaining-Based Load Control Mechanism Design for Microgrids
On average, vehicles only use 4% of the day for commuting and 96% idle parking in a parking lot. Coordinated EV charging and discharging has the potential to enable peak shaving, valley filling and load curving in microgrids. Noncooperative game helps the players making decisions selfishly and keeps privacy for each player, however, the obtained solution may not be social optimal and cannot guarantee a fair resource allocation. Instead, the cooperative game theory enables the players to form coalitions and share information to improve the Pareto-optima and social-optima. In the majority of existing works on V2G load control, the effect of shared information in the coalition has not been addressed fully. By forming a coalition and exchanging information with each other, players get a better understanding of the current situation and have the potential to gain more benefits. We explored energy trading with shared information and proposed an overload penalty distribution method to realize load control for the microgrid.
Goals of the project: Form a cooperative game to address the energy trading problem in the microgrids and propose a relaxed constraint set for the game considering shared information. Players in the game will collaborate together to maintain an ideal load profile while gain extra benefits.
5.2 : Real-time Load Control Mechanism Design for Microgrids
In most existing work on V2G load control, the effect of unexpected power flow fluctuation and uncertain user behaviors on load profile has not been addressed fully. Numerous recent studies concentrate on forecasting the day-ahead load profile using distinct mathematical models and real-world data. Nevertheless, these methods encounter challenges when faced with abrupt shifts, such as notable fluctuations in the main grid power system or the unanticipated arrival or departure of a considerable number of electric vehicles (EVs), disrupting the charging station’s (CS) load profile. This disturbance can result in significant fluctuations in energy demand, leading to extra costs. The conventional load control method, reliant on the day-ahead predicted load profile, proves insufficient for the CS to adjust its charging quotas adequately in response to such fluctuations.
Goals of the project: In addressing unforeseen fluctuations in the charging station (CS) load and enhancing information provision for electric vehicles (EVs), we explore the energy trading within microgrids using time-series game theoretic methods, a fusion of game theory and the sliding window approach. The utilization of the sliding window allows players to acquire real-time insights into changes, enabling them to counteract any adverse effects effectively.
5.3 : System decomposition based on game theoretic method
To address the energy trading problem in microgrids with multiple users who share different interests, the objective function of game theoretic method usually contains a lot of components. Especially for the Nash Bargaining game, the multiplication of player’s utility function is used as the cost function which makes the calculation much harder to solve. To reduce the computational burden, some decomposition method has been applied to the game theoretic method. The main idea of decomposition is to break the major problem into several sub-problems and solve the sub-problems instead. Usually, the sub-problems only contain the variables that belongs to one party which makes the optimization easier to be solved. However, how to decouple those variables to make the calculation more efficient is still hard to answer.
Research Area 6 : Motion Planning and Computer Vision for F1 Tenth vehicle
This project centers around the development and advancement of small-scale, 1/10th size Formula 1 race cars that are fully autonomous. It provides a unique platform for researchers, engineers, and enthusiasts to explore and implement cutting-edge algorithms in perception, decision-making, and control, all within the challenging context of high-speed, dynamic racing environments. The F1Tenth vehicle’s base code and construction are based on F1tenth.
The key areas of research are motion planning and computer vision. The GitHub repository for existing work can be found at GitHub F1tenth-UFL. A few sample motion planning algorithms are illustrated below.
Research Area 7 : Estimation, Control, and Health Monitoring of Energy Storage Systems
Batteries are one of the key enablers for modern sustainable energy in the power grid and transportation systems. Large batteries are utilized in power grids, HEVs, and plugin hybrid electric vehicles (PHEVs) as the main source of energy storage. Intelligent management of batteries utilizing the knowledge of the characteristics of the battery can potentially lead to improved energy management in the power grid and transportation systems. In Li-ion batteries, the main challenge lies in the lack of real-time accessible information which is crucial for accurate management. This part of my research focused on different types of battery faults and resulted in algorithms that detect, isolate, and estimate some of the fault characteristics that can be utilized for fault-tolerant energy management systems. In particular, I proposed algorithms for i) diagnosis of faults in battery current, voltage, temperature sensor, fan actuator failure, and disturbance in battery state of charge (SoC); ii) diagnosis of thermal faults to prevent several thermal failures like thermal runaway.
Goal of this project: The objective of this research is development of advanced state-of-the-art battery management features by proposing novel estimation and diagnostic algorithms. Also, the diagnosis of thermal faults e.g. thermal runaway is one of the first works in battery diagnostics that estimates the magnitude of effects of specific faults in batteries.
Testing videos:
