| Part of a series on statistics | |
| Probability theory | |
|---|---|
 | |
IntroductioneditA stochastic or random process can be defined as a collection of random variables that is indexed by some mathematical set, meaning that each random variable of the stochastic process is uniquely associated with an element in the set. The set used to index the random variables is called the index set. Historically, the index set was some subset of the real line, such as the natural numbers, giving the index set the interpretation of time. Each random variable in the collection takes values from the same mathematical space known as the state space. This state space can be, for example, the integers, the real line or n-dimensional vector process or ℝn or a manifold. NotationeditA stochastic process can be denoted, among other ways, by {Xt} X(t) even though it is an abuse of function notation. For example, t, and not the entire stochastic process. If the index set is p and zero with probability 1−p, so the index set of this random walk is the natural numbers, while its state space is the integers. If The Wiener process is a stochastic process with stationary and independent increments that are normally distributed based on the size of the increments. The Wiener process is named after Norbert Wiener, who proved its mathematical existence, but the process is also called the Brownian motion process or just Brownian motion due to its historical connection as a model for Brownian movement in liquids.  Realizations of Wiener processes (or Brownian motion processes) with drift (blue) and without drift (μ, which is a real number, then the resulting stochastic process is said to have drift
The Poisson process is a stochastic process that has different forms and definitions. It can be defined as a counting process, which is a stochastic process that represents the random number of points or events up to some time. The number of points of the process that are located in the interval from zero to some given time is a Poisson random variable that depends on that time and some parameter. This process has the natural numbers as its state space and the non-negative numbers as its index set. This process is also called the Poisson counting process, since it can be interpreted as an example of a counting process. If a Poisson process is defined with a single positive constant, then the process is called a homogeneous Poisson process. The homogeneous Poisson process is a member of important classes of stochastic processes such as Markov processes and Lévy processes. The homogeneous Poisson process can be defined and generalized in different ways. It can be defined such that its index set is the real line, and this stochastic process is also called the stationary Poisson process. If the parameter constant of the Poisson process is replaced with some non-negative integrable function of (Ω,ℱ,P), where σ-algebra, and S, which must be measurable with respect to some (Ω,ℱ,P) and a measurable space {X(t):t∈T}. Historically, in many problems from the natural sciences a point t. A stochastic process can also be written as ω∈Ω. There are other ways to consider a stochastic process, with the above definition being considered the traditional one. For example, a stochastic process can be interpreted or defined as a T into the space T the interpretation of time. In addition to these sets, the index set n-dimensional Euclidean space, where an element n-dimensional Euclidean spaces, complex planes, or more abstract mathematical spaces. The state space is defined using elements that reflect the different values that the stochastic process can take. Sample functioneditA sample function is a single outcome of a stochastic process, so it is formed by taking a single possible value of each random variable of the stochastic process. More precisely, if X(⋅,ω):T→S, is called a sample function, a realization, or, particularly when ω∈Ω, there exists a sample function that maps the index set {X(t):t∈T} is a stochastic process with state space t1∈[0,∞) and Xt2−Xt1 is a n-dimensional Euclidean space or more abstract spaces such as Banach spaces. Further definitionseditLaweditFor a stochastic process X is defined as the pushforward measure: ∘ denotes function composition and X, where t∈T, so the law of a stochastic process is a probability measure.
For a measurable subset X gives μ(B)=P({ω∈Ω:X(ω)∈B}).
The law of a stochastic process or a random variable is also called the probability law, probability distribution, or the distribution. Finite-dimensional probability distributionseditThis measure μ onto a finite subset of n-fold Cartesian power μt1,…,tn(C)=P({ω∈Ω:(Xt1(ω),…,Xtn(ω))∈C}). The finite-dimensional distributions of a stochastic process satisfy two mathematical conditions known as consistency conditions. Stationarityeditall have the same probability distribution. The index set of a stationary stochastic process is usually interpreted as time, so it can be the integers or the real line. But the concept of stationarity also exists for point processes and random fields, where the index set is not interpreted as time. When the index set X has a finite second moment for all Xt+h depends only on the number {ℱt}t∈T, on a probability space s≤t, where T. With the concept of a filtration, it is possible to study the amount of information contained in a stochastic process t. The intuition behind a filtration Xt is known or available, which is captured in X that has the same index set (Ω,ℱ,P) as another stochastic process t∈T the following X and T and set space X and Y are indistinguishable.
SeparabilityeditSeparability is a property of a stochastic process based on its index set in relation to the probability measure. The property is assumed so that functionals of stochastic processes or random fields with uncountable index sets can form random variables. For a stochastic process to be separable, in addition to other conditions, its index set must be a separable space, which means that the index set has a dense countable subset. More precisely, a real-valued continuous-time stochastic process T has a dense countable subset P(Ω0)=0, such that for every open set {Xt∈F for all t∈G∩U} and n-dimensional Euclidean space. The concept of separability of a stochastic process was introduced by Joseph Doob. The underlying idea of separability is to make a countable set of points of the index set determine the properties of the stochastic process. Any stochastic process with a countable index set already meets the separability conditions, so discrete-time stochastic processes are always separable. A theorem by Doob, sometimes known as Doob’s separability theorem, says that any real-valued continuous-time stochastic process has a separable modification. Versions of this theorem also exist for more general stochastic processes with index sets and state spaces other than the real line. IndependenceeditTwo stochastic processes (Ω,ℱ,P) with the same index set t1,…,tn∈T, the random vectors {Xt} and {Xt},{Yt} uncorrelated⟺K𝐗𝐘(t1,t2)=0∀t1,t2. Independence implies uncorrelatednesseditIf two stochastic processes {Xt} and {Xt},{Yt} orthogonal⟺R𝐗𝐘(t1,t2)=0∀t1,t2. Skorokhod spaceeditMartingales can also be created from stochastic processes by applying some suitable transformations, which is the case for the homogeneous Poisson process (on the real line) resulting in a martingale called the compensated Poisson process. Martingales can also be built from other martingales. For example, there are martingales based on the martingale the Wiener process, forming continuous-time martingales. Martingales mathematically formalize the idea of a 'fair game' where it is possible form reasonable expectations for payoffs, and they were originally developed to show that it is not possible to gain an 'unfair' advantage in such a game. But now they are used in many areas of probability, which is one of the main reasons for studying them. Many problems in probability have been solved by finding a martingale in the problem and studying it. Martingales will converge, given some conditions on their moments, so they are often used to derive convergence results, due largely to martingale convergence theorems. Martingales have many applications in statistics, but it has been remarked that its use and application are not as widespread as it could be in the field of statistics, particularly statistical inference. They have found applications in areas in probability theory such as queueing theory and Palm calculus and other fields such as economics and finance. Lévy processeditn-dimensional Euclidean space. Other stochastic processes such as renewal and counting processes are studied in the theory of point processes.
HistoryeditEarly probability theoryeditProbability theory has its origins in games of chance, which have a long history, with some games being played thousands of years ago, but very little analysis on them was done in terms of probability. The year 1654 is often considered the birth of probability theory when French mathematicians Pierre Fermat and Blaise Pascal had a written correspondence on probability, motivated by a gambling problem. But there was earlier mathematical work done on the probability of gambling games such as Liber de Ludo Aleae by Gerolamo Cardano, written in the 16th century but posthumously published later in 1663. After Cardano, Jakob Bernoulli wrote Ars Conjectandi, which is considered a significant event in the history of probability theory. Bernoulli's book was published, also posthumously, in 1713 and inspired many mathematicians to study probability. But despite some renowned mathematicians contributing to probability theory, such as Pierre-Simon Laplace, Abraham de Moivre, Carl Gauss, Siméon Poisson and Pafnuty Chebyshev, most of the mathematical community did not consider probability theory to be part of mathematics until the 20th century. Statistical mechanicseditIn the physical sciences, scientists developed in the 19th century the discipline of statistical mechanics, where physical systems, such as containers filled with gases, are regarded or treated mathematically as collections of many moving particles. Although there were attempts to incorporate randomness into statistical physics by some scientists, such as Rudolf Clausius, most of the work had little or no randomness. This changed in 1859 when James Clerk Maxwell contributed significantly to the field, more specifically, to the kinetic theory of gases, by presenting work where he modelled the gas particles as moving in random directions at random velocities. The kinetic theory of gases and statistical physics continued to be developed in the second half of the 19th century, with work done chiefly by Clausius, Ludwig Boltzmann and Josiah Gibbs, which would later have an influence on Albert Einstein's mathematical model for Brownian movement. Measure theory and probability theoryeditAt the International Congress of Mathematicians in Paris in 1900, David Hilbert presented a list of mathematical problems, where his sixth problem asked for a mathematical treatment of physics and probability involving axioms. Around the start of the 20th century, mathematicians developed measure theory, a branch of mathematics for studying integrals of mathematical functions, where two of the founders were French mathematicians, Henri Lebesgue and Émile Borel. In 1925, another French mathematician Paul Lévy published the first probability book that used ideas from measure theory. In the 1920s, fundamental contributions to probability theory were made in the Soviet Union by mathematicians such as Sergei Bernstein, Aleksandr Khinchin, and Andrei Kolmogorov. Kolmogorov published in 1929 his first attempt at presenting a mathematical foundation, based on measure theory, for probability theory. In the early 1930s, Khinchin and Kolmogorov set up probability seminars, which were attended by researchers such as Eugene Slutsky and Nikolai Smirnov, and Khinchin gave the first mathematical definition of a stochastic process as a set of random variables indexed by the real line. Birth of modern probability theoryeditIn 1933, Andrei Kolmogorov published in German, his book on the foundations of probability theory titled Grundbegriffe der Wahrscheinlichkeitsrechnung, where Kolmogorov used measure theory to develop an axiomatic framework for probability theory. The publication of this book is now widely considered to be the birth of modern probability theory, when the theories of probability and stochastic processes became parts of mathematics. After the publication of Kolmogorov's book, further fundamental work on probability theory and stochastic processes was done by Khinchin and Kolmogorov as well as other mathematicians such as Joseph Doob, William Feller, Maurice Fréchet, Paul Lévy, Wolfgang Doeblin, and Harald Cramér. Decades later, Cramér referred to the 1930s as the "heroic period of mathematical probability theory". World War II greatly interrupted the development of probability theory, causing, for example, the migration of Feller from Sweden to the United States of America and the death of Doeblin, considered now a pioneer in stochastic processes.  Mathematician Joseph Doob did early work on the theory of stochastic processes, making fundamental contributions, particularly in the theory of martingales. His book Stochastic Processes is considered highly influential in the field of probability theory.
Stochastic processes after World War IIeditAfter World War II, the study of probability theory and stochastic processes gained more attention from mathematicians, with significant contributions made in many areas of probability and mathematics as well as the creation of new areas. Starting in the 1940s, Kiyosi Itô published papers developing the field of stochastic calculus, which involves stochastic integrals and stochastic differential equations based on the Wiener or Brownian motion process. Also starting in the 1940s, connections were made between stochastic processes, particularly martingales, and the mathematical field of potential theory, with early ideas by Shizuo Kakutani and then later work by Joseph Doob. Further work, considered pioneering, was done by Gilbert Hunt in the 1950s, connecting Markov processes and potential theory, which had a significant effect on the theory of Lévy processes and led to more interest in studying Markov processes with methods developed by Itô. In 1953, Doob published his book Stochastic processes, which had a strong influence on the theory of stochastic processes and stressed the importance of measure theory in probability. Doob also chiefly developed the theory of martingales, with later substantial contributions by Paul-André Meyer. Earlier work had been carried out by Sergei Bernstein, Paul Lévy and Jean Ville, the latter adopting the term martingale for the stochastic process. Methods from the theory of martingales became popular for solving various probability problems. Techniques and theory were developed to study Markov processes and then applied to martingales. Conversely, methods from the theory of martingales were established to treat Markov processes. Other fields of probability were developed and used to study stochastic processes, with one main approach being the theory of large deviations. The theory has many applications in statistical physics, among other fields, and has core ideas going back to at least the 1930s. Later in the 1960s and 1970s, fundamental work was done by Alexander Wentzell in the Soviet Union and Monroe D. Donsker and Srinivasa Varadhan in the United States of America, which would later result in Varadhan winning the 2007 Abel Prize. In the 1990s and 2000s the theories of Schramm–Loewner evolution and rough paths were introduced and developed to study stochastic processes and other mathematical objects in probability theory, which respectively resulted in Fields Medals being awarded to Wendelin Werner in 2008 and to Martin Hairer in 2014. The theory of stochastic processes still continues to be a focus of research, with yearly international conferences on the topic of stochastic processes. Discoveries of specific stochastic processeseditAlthough Khinchin gave mathematical definitions of stochastic processes in the 1930s, specific stochastic processes had already been discovered in different settings, such as the Brownian motion process and the Poisson process. Some families of stochastic processes such as point processes or renewal processes have long and complex histories, stretching back centuries. Bernoulli processeditThe Bernoulli process, which can serve as a mathematical model for flipping a biased coin, is possibly the first stochastic process to have been studied. The process is a sequence of independent Bernoulli trials, which are named after Jacob Bernoulli who used them to study games of chance, including probability problems proposed and studied earlier by Christiaan Huygens. Bernoulli's work, including the Bernoulli process, were published in his book Ars Conjectandi in 1713. Random walkseditIn 1905, Karl Pearson coined the term random walk while posing a problem describing a random walk on the plane, which was motivated by an application in biology, but such problems involving random walks had already been studied in other fields. Certain gambling problems that were studied centuries earlier can be considered as problems involving random walks. For example, the problem known as the Gambler's ruin is based on a simple random walk, and is an example of a random walk with absorbing barriers. Pascal, Fermat and Huyens all gave numerical solutions to this problem without detailing their methods, and then more detailed solutions were presented by Jakob Bernoulli and Abraham de Moivre. For random walks in T include:
where the symbol ∈ can be read "a member of the set", as in n-dimensional Euclidean space. ApplicationeditApplications in FinanceeditBlack-Scholes ModeleditOne of the most famous applications of stochastic processes in finance is the Black-Scholes model for option pricing. Developed by Fischer Black, Myron Scholes, and Robert Merton (whose contributions led to the 1997 Nobel Memorial Prize in Economic Sciences), this model uses Geometric Brownian motion, a specific type of stochastic process, to describe the dynamics of asset prices. The model assumes that the price of a stock follows a continuous-time stochastic process driven by a stochastic differential equation (SDE): μ is the drift rate, representing the expected return of the asset.
|