quantize_pdf_to_cdf

rwkvzip::coders::ac

Function quantize_pdf_to_cdf 

Source 
pub fn quantize_pdf_to_cdf(pdf: &[f64]) -> Vec<u32>
Expand description

Quantize probability distribution to integer CDF.

The CDF is constructed to be monotonically non-decreasing with:

  • cdf[0] = 0
  • cdf[vocab_size] = CDF_TOTAL

§Arguments

  • pdf - Probability distribution (must sum to ~1.0)

§Returns

Integer CDF with length vocab_size + 1