New Advances in Uncertainty Analysis and Estimation
Organizer: Puneet Singla (SUNY Buffalo), Raktim Bhattacharya (Texas A & M University)
This workshop would cover topics from basic linear and nonlinear stochastic processes to well-known Kalman filtering methods to recently developed nonlinear estimation methods at a level of detail compatible with the design and implementation of modern control and estimation of dynamical systems. These diverse topics will be covered in an integrated fashion, using a framework derived from stochastic processes, estimation, control, and approximation theory. The reliability and limitations of various methods discussed will be assessed by considering various academic and engineering problems.
- Short Review of Probability and Stochastic Processes (Speaker: Puneet Singla)
Abstract: This talk will review basic concepts related to random variables, random process, conditional probability density function, Bayesian inference, entropy, Kullback-Fisher information, continuous and discrete random process, Brownian motion, and white noise. These will provide the necessary mathematical background for the material presented later in the workshop.
- Propagation of PDF by solving Kolmogorov Equation (Speaker: Puneet Singla, Raktim Bhattacharya)
Abstract: This talk will focus on recent development in computational methods for uncertainty characterization and forecasting for nonlinear systems. The central idea is to replace evolution of initial conditions for a large dynamical system by evolution of probability density functions (pdf) for state variables. The use of Fokker-Planck-Kolmogorov equation (FPKE) and Chapman-Kolmogorov equation (CKE) to determine evolution of state pdf due to probabilistic uncertainty in initial or boundary conditions, model parameters and forcing function will be discussed. Analytical solutions for the FPKE/CKE exist only for a stationary pdf and are restricted to a limited class of dynamical systems. Traditional numerical approaches based upon variational formulation which discretize the space in which the pdf lies, suffer from the “curse of dimensionality.” In this talk, we will discuss that how one can make use of recent advances in approximation theory to not only break the “curse of dimensionality” but can also pose the the pdf evolution problem as a convex optimization problem with guaranteed convergence. In particular, the audience will be introduced to use of Gaussian mixture model based approaches to solve both the differential and integral form of the Kolmogorov equation.
For systems with weak diffusion, the FPKE can be simplified to the continuity equation. This results in considerable simplification in the solution of the governing equation. The continuity equation being first order linear can be solved using method of characteristics and Rothe’s method. Additionally, we will describe use of maximum entropy basis functions to approximate the PDF evolution in a mesh less computational framework.
- Spectral Representation and Moment Propagation (Speaker: Raktim Bhattacharya)
Abstract: This talk will discuss approximation of random processes using KL and polynomial chaos expansions, and describe when these approximation techniques can be applied. We will discuss how these techniques can be applied to both linear and nonlinear systems. We will also highlight the computational complexity associated with polynomial chaos approximation, in particular with Galerkin projections, and make a case for stochastic collocation techniques. Examples illustrating strengths and weakness of spectral approximations will also be presented. Implementation details and accuracy of approximations will be discussed using few numerical examples.
- Quadrature methods (Speaker: Puneet Singla)
Abstract: This talk will introduce the theory of Gauss quadrature methods to evaluate expectation integral involving a generic density function. Quadrature methods involve an approximation of the expectation integral as a weighted sum of integrand values at specified points within the domain of integration. A quadrature rule is said to be exact to degree d, if it can only integrate all polynomials with degree ≤ d. For 1-D (1-Dimensional) integrals, one needs N quadrature points according to the Gaussian quadrature scheme to exactly reproduce the expectation integrals of polynomials with degree 2N-1 or less. However, in generic n-D, one needs to take the tensor product of 1-D quadrature points and hence would yield a total of Nn quadrature points. This is a non-trivial number of points that might make the calculation of the integral computationally expensive, especially when the evaluation of function at each cubature point itself can be an expensive procedure. This talk will introduce recently developed Conjugate Unscented Transformation (CUT) approach to accurately evaluate expectation integrals in high dimension space while minimizing the number of simulations. Rather than using tensor products as in Gauss quadrature, the CUT approach judiciously selects speciﬁc structures to extract symmetric quadrature points. Several benchmark problems will be considered to highlight relative merits of various algorithms and their use in stochastic collocation.
- Applications to Estimation & Filtering (Speakers: Puneet Singla and Raktim Bhattacharya)
Abstract: This talk will introduce the concept of model-data fusion, which has its birth with the development of Kalman filter for linear system. We will discuss that how various uncertainty propagation methods introduced in prior sections can be used along with Bayes’ rule to find system state and parameter estimates. In addition, the concept of maximum likelihood estimation and best linear unbiased estimator will be discussed. Relative merits of different approaches will be discussed while considering various benchmark problems.