Inverse Transform Sampling

An introduction to Inverse Transform Sampling

Under development

<!--
  See https://www.debugbear.com/blog/responsive-images#w-descriptors-and-the-sizes-attribute and
  https://developer.mozilla.org/en-US/docs/Learn/HTML/Multimedia_and_embedding/Responsive_images for info on defining 'sizes' for responsive images
-->

  <source
    class="responsive-img-srcset"
    srcset="/assets/img/statistics/ITS/cdf-pdf-480.webp 480w,/assets/img/statistics/ITS/cdf-pdf-800.webp 800w,/assets/img/statistics/ITS/cdf-pdf-1400.webp 1400w,"
    
      sizes="95vw"
    
    type="image/webp"
  >

<img
  src="/assets/img/statistics/ITS/cdf-pdf.png"
  
    class="img-fluid rounded z-depth-1"
  
  
    width="100%"
  
  
    height="auto"
  
  
  
  
    title="example image"
  
  
  
    loading="eager"
  
  onerror="this.onerror=null; $('.responsive-img-srcset').remove();"
>

</picture>

</figure>

</div>

</div>

Relation between CDF and PDF.
\[\lim_{x \to +\infty} F_X(x)=1\] \[f_X(x) = \frac{d}{dx} F_X(x)\]

Using the CDF instead of the PDF provides a bijective relationship between $[0,1]$ and $[a,b]$. Without the CDF, a value like 0.2 could correspond to multiple values in $[a,b]$, making transformation ambiguous.

How Inverse Transform Sampling Works

In Inverse Transform Sampling, we generate a random number $U$ uniformly distributed over $[0,1]$ and apply the inverse CDF ($F^{-1}$) to obtain a corresponding value $X$ in the target interval $[a,b]$:

\[X = F^{-1}(U)\]

The key insight is that the CDF increases when the PDF is nonzero and remains constant when the PDF is zero. If we sample uniformly along the y-axis from $[0,1]$, we are more likely to land in a region where the CDF is increasing rather than where it is flat. This ensures that our samples are more frequently drawn from regions where the PDF is nonzero, improving the efficiency of sampling.

If we were to use the PDF directly, we would not have a unique mapping between $U$ and $X$ because the PDF represents density rather than a one-to-one correspondence. However, since the CDF is strictly increasing for a continuous distribution, each $U$ in $[0,1]$ corresponds uniquely to a single $X$ in $[a,b]$, ensuring invertibility.

Thus, without the CDF, multiple values of $X$ could correspond to the same $U$, making the transformation ambiguous. The CDF guarantees a well-defined inverse mapping, allowing us to efficiently sample from the target distribution.

From the law of large number we know that the integral $I=\int f(x)$ [!NOTE]

Law of large numbers > The law of large numbers tells us that if we take a sequence of independent and identically distributed (iid) random variables $X_i$ with an expectation $\mathbb{E}[X]$, then the empirical mean converges almost surely to the expectation: > $\frac{1}{N} \sum_{i=1}^{N} X_i \quad \xrightarrow{N \to \infty} \quad \mathbb{E}[X]$

–>

Previous: Monte Carlo Simulation