In all likelihood I’ll post here a random selection of topics I’m interested in (machine learning, statistics and possible some simple mathematics). Partly to force myself to work out “clean” arguments. Even better if some posts are of use for other people.
Measurable Estimator Selection
In statistics or machine learning we estimate parameters/functions from data and analyze the estimate, for instance by providing confidence regions around it. To be able to analyze the estimate it is obviously useful to be able to use expressions of the form , where is some region in parameter space. Generally we have some inference method that takes an experiment and assigns a parameter based on the experiment. Calling this function we have and we need to know that is measurable, or in other words that the estimator selection is measurable.
Let be some probability space with i.i.d. random variables defined on it and some parameter space from which we want to choose the element that maximizes the cost function , where attains values in . is called an M-estimator and one might ask when the selection is measurable, where .
The most natural setting seems here to be a compact topological space equipped with the Borel-algebra and continuous with a unique maximum for every and measurable as a map for every . The maximum is then well defined and is measurable since the maximum can be reduced to a maximum over a countable set which then implies directly measurabilty of the map. Also the selection is measurable. Let be an open set in and a countable dense subset of . Then, using the continuity of ,
This extends then directly to arbitrary Borel-measurable subsets of .