Inspection

kudio.core.report

What can be said about a recording with no clean reference to compare it to.

Every quality metric in kudio.core.evaluator -- SNR, SI-SDR, PESQ, STOI -- needs the clean signal alongside the degraded one. Synthetic datasets have that. Real recordings never do, and those are the ones you actually need to judge: score a denoised field recording against the noisy original and you measure how much you changed it, not whether it got better.

This module is the half of that gap which costs nothing: level, clipping, DC offset, how much of the file is silence, where the noise floor sits, a crude SNR estimate, and how much bandwidth the content really occupies. None of it predicts a listener's opinion -- that needs a trained model such as DNSMOS or NISQA -- but all of it catches the mistakes that make a dataset quietly useless before anyone gets as far as an opinion.

Classes:

Name	Description
AudioReport	Reference-free measurements of one clip. See audio_report.

Functions:

Name	Description
audio_report	Inspect y without needing a clean reference to compare it against.

AudioReport dataclass

AudioReport(
    sr: int,
    duration: float,
    peak_dbfs: float,
    rms_dbfs: float,
    lufs: float,
    clipped_samples: int,
    clipped_ratio: float,
    dc_offset: float,
    silence_ratio: float,
    noise_floor_dbfs: float,
    estimated_snr_db: float,
    bandwidth_hz: float,
    band_limited: bool,
    non_finite_samples: int = 0,
)

Reference-free measurements of one clip. See audio_report.

Methods:

Name	Description
problems	The findings worth acting on, in plain words. Empty is good news.

Attributes:

Name	Type	Description
peak_dbfs	float	dBFS of the loudest sample; -inf for digital silence
lufs	float	BS.1770 integrated loudness, or nan when the clip is too short
clipped_samples	int	samples inside a run of at least three pinned to full scale
dc_offset	float	mean sample value; a healthy recording sits at zero
estimated_snr_db	float	loud frames against quiet frames -- an estimate, not a measurement
bandwidth_hz	float	highest frequency still carrying content
band_limited	bool	content stops well below Nyquist: upsampled, or through a narrow codec
non_finite_samples	int	samples that are NaN or inf; with any of these the rest is meaningless

peak_dbfs instance-attribute

peak_dbfs: float

dBFS of the loudest sample; -inf for digital silence

lufs instance-attribute

lufs: float

BS.1770 integrated loudness, or nan when the clip is too short

clipped_samples instance-attribute

clipped_samples: int

samples inside a run of at least three pinned to full scale

dc_offset instance-attribute

dc_offset: float

mean sample value; a healthy recording sits at zero

estimated_snr_db instance-attribute

estimated_snr_db: float

loud frames against quiet frames -- an estimate, not a measurement

bandwidth_hz instance-attribute

bandwidth_hz: float

highest frequency still carrying content

band_limited instance-attribute

band_limited: bool

content stops well below Nyquist: upsampled, or through a narrow codec

non_finite_samples class-attribute instance-attribute

non_finite_samples: int = 0

samples that are NaN or inf; with any of these the rest is meaningless

problems

problems() -> List[str]

The findings worth acting on, in plain words. Empty is good news.

Source code in kudio/core/report.py

def problems(self) -> List[str]:
    """The findings worth acting on, in plain words. Empty is good news."""
    found = []
    if self.non_finite_samples:
        # First, and on its own: every number in this report came out of
        # the same samples, so with NaN among them there is nothing else
        # here worth reading. It has to be its own test rather than
        # something the tests below catch, because a comparison against
        # nan is false whichever way round it is written -- which is how a
        # corrupt clip came to be reported as a muted microphone, the one
        # answer that stops anybody looking any further.
        return [f"{self.non_finite_samples} samples are not finite (NaN "
                f"or inf) — the clip is corrupt, not quiet, and every "
                f"measurement here is meaningless. Repair it or drop it "
                f"from the set"]
    if self.peak_dbfs == float('-inf'):
        return ["the clip is digital silence"]
    if self.clipped_samples:
        found.append(
            f"{self.clipped_samples} samples clipped "
            f"({self.clipped_ratio * 100:.2f}% of the clip) — the recording "
            f"was already too hot; no processing gets that back")
    elif self.peak_dbfs > 0.0:
        found.append(f"peaks at {self.peak_dbfs:+.1f} dBFS, past full scale — "
                     f"it will clip the moment it is written to an integer "
                     f"format")
    elif self.peak_dbfs > -0.5:
        # + 0.0 so a peak of 32767/32768 prints as 0.0, not -0.0
        found.append(f"peaks at {round(self.peak_dbfs, 1) + 0.0:.1f} dBFS, "
                     f"right against the "
                     f"ceiling — one more gain stage will clip it")
    if self.peak_dbfs < -30.0:
        found.append(f"peaks at only {self.peak_dbfs:.1f} dBFS — very quiet, "
                     f"so the usable bit depth is small")
    if abs(self.dc_offset) > 0.01:
        found.append(f"DC offset of {self.dc_offset:+.3f} — the waveform is "
                     f"not centred on zero, which wastes headroom")
    if self.band_limited:
        source = _nearest_common_rate(2 * self.bandwidth_hz, below=self.sr)
        came_from = (f"upsampled from {source} Hz" if source
                     else f"band-limited to about {self.bandwidth_hz:.0f} Hz")
        found.append(
            f"content stops at {self.bandwidth_hz:.0f} Hz although the file "
            f"claims {self.sr} Hz — it was very likely {came_from}, and the "
            f"extra rate is empty")
    if self.silence_ratio > 0.6:
        found.append(f"{self.silence_ratio * 100:.0f}% of the clip is silence")
    if np.isfinite(self.estimated_snr_db) and self.estimated_snr_db < 10.0:
        found.append(f"estimated SNR around {self.estimated_snr_db:.0f} dB — "
                     f"the noise floor is close to the content")
    return found

audio_report

audio_report(y: ndarray, sr: int) -> AudioReport

Inspect y without needing a clean reference to compare it against.

Mono only -- mix down first if you have channels; the questions this answers are about the recording, not the mix.

>>> report = kudio.audio_report(y, sr)
>>> for problem in report.problems():
...     print(problem)
content stops at 3844 Hz although the file claims 16000 Hz — ...

A clip carrying NaN or inf is measured and counted rather than refused: the count is the finding, reported first by AudioReport.problems. The peak, RMS and loudness come out nan; the noise floor, the SNR estimate and the silence share are taken over the frames that are finite. Inspecting a corpus is exactly when you want to be told which files are broken, not stopped at the first one.

The noise floor is read from frames with the DC offset taken out, so an offset is reported as an offset rather than as noise. When at least a tenth of the frames are digital silence, the floor is digital silence: -inf, with an SNR estimate of inf.

Clipping is counted in runs of three or more pinned samples; a run of one or two is indistinguishable from a peak that touched full scale.

Raises:

Type	Description
FeatureError	on a non-mono array or a non-positive rate.

Source code in kudio/core/report.py

def audio_report(y: np.ndarray, sr: int) -> AudioReport:
    """Inspect *y* without needing a clean reference to compare it against.

    Mono only -- mix down first if you have channels; the questions this
    answers are about the recording, not the mix.

    >>> report = kudio.audio_report(y, sr)               # doctest: +SKIP
    >>> for problem in report.problems():                # doctest: +SKIP
    ...     print(problem)
    content stops at 3844 Hz although the file claims 16000 Hz — ...

    A clip carrying NaN or inf is measured and counted rather than refused:
    the count is the finding, reported first by :meth:`AudioReport.problems`.
    The peak, RMS and loudness come out nan; the noise floor, the SNR estimate
    and the silence share are taken over the frames that are finite. Inspecting
    a corpus is exactly when you want to be told which files are broken, not
    stopped at the first one.

    The noise floor is read from frames with the DC offset taken out, so an
    offset is reported as an offset rather than as noise. When at least a
    tenth of the frames are digital silence, the floor *is* digital silence:
    ``-inf``, with an SNR estimate of ``inf``.

    Clipping is counted in runs of three or more pinned samples; a run of one
    or two is indistinguishable from a peak that touched full scale.

    :raises FeatureError: on a non-mono array or a non-positive rate.
    """
    if sr <= 0:
        raise FeatureError(f"audio_report() needs a positive sample rate, got {sr}")
    y = np.ascontiguousarray(np.asarray(as_float(y), dtype=np.float64).squeeze())
    if y.ndim != 1:
        raise FeatureError(f"audio_report() takes mono audio, got shape {y.shape}")
    if y.size == 0:
        raise FeatureError("audio_report() got an empty waveform")

    non_finite = int(np.count_nonzero(~np.isfinite(y)))
    peak = float(np.max(np.abs(y)))
    rms = float(np.sqrt(np.mean(y ** 2)))
    clipped = _clipped_run_count(y)

    size = max(16, int(round(_FRAME * sr)))
    # the offset out first: left in, a 0.03 offset read as a noise floor of
    # -30 dBFS on a take whose real floor was -57
    usable = np.isfinite(y)
    centred = y - (float(np.mean(y[usable])) if usable.any() else 0.0)
    frame_rms = np.sqrt(np.mean(_frame(centred, size, max(1, int(round(_HOP * sr))))
                                ** 2, axis=-1))
    with np.errstate(divide='ignore', invalid='ignore'):
        frame_db = 20.0 * np.log10(np.maximum(frame_rms, 0.0))

    finite = frame_db[np.isfinite(frame_db)]
    # exact zeros are found on the samples as they are: with the mean taken
    # out, a zeroed pause is a tiny constant rather than nothing
    hop = max(1, int(round(_HOP * sr)))
    zero_frames = np.all(_frame(y, size, hop) == 0.0, axis=-1)
    digital_silence = float(np.mean(zero_frames)) if zero_frames.size else 0.0
    if finite.size:
        loud = float(np.percentile(finite, 95))
        floor = float(np.percentile(finite, 10))
        if digital_silence >= 0.1:
            # pauses of exact zeros: the floor is silence, not the quietest
            # stretch of speech, which is what the percentile used to land on
            floor = float('-inf')
        silence_ratio = float(np.mean((frame_db < loud - _SILENCE_REL_DB)
                                      | zero_frames))
        snr = loud - floor
    else:                                       # every frame is exact silence
        loud = floor = float('-inf')
        silence_ratio, snr = 1.0, float('-inf')

    if non_finite:
        # undefined, and loudness() would warn about the samples this report
        # already names as its first finding
        lufs = float('nan')
    else:
        try:
            lufs = loudness(y, sr)
        except FeatureError:                    # shorter than a BS.1770 block
            lufs = float('nan')

    bandwidth = _bandwidth(y, sr)
    nyquist = sr / 2.0

    return AudioReport(
        sr=int(sr),
        duration=len(y) / sr,
        peak_dbfs=_db(peak),
        rms_dbfs=_db(rms),
        lufs=float(lufs),
        clipped_samples=clipped,
        clipped_ratio=clipped / len(y),
        dc_offset=float(np.mean(y)),
        silence_ratio=silence_ratio,
        noise_floor_dbfs=floor,
        estimated_snr_db=snr,
        bandwidth_hz=bandwidth,
        band_limited=bool(0.0 < bandwidth < _BAND_LIMITED_RATIO * nyquist),
        non_finite_samples=non_finite,
    )

kudio.core.loudness

Perceptual loudness to ITU-R BS.1770-4, and normalisation based on it.

normalize and normalize_db scale by the peak sample, which says nothing about how loud something sounds: a clip with one stray transient normalises to a whisper. BS.1770 answers the question peak normalisation cannot -- how loud is this to a listener -- by K-weighting the signal, measuring mean square over 400 ms blocks, and gating away the silence between words so a pause does not drag the number down.

Why this matters beyond metering: an A/B comparison between two versions of a clip is not valid unless they are loudness matched. The louder one wins, whatever its quality. match_loudness is the primitive for that.

The standard publishes its K-weighting coefficients at 48 kHz only. Speech work mostly happens at 8 or 16 kHz, so the filters here are re-discretised to the requested rate through a prewarped bilinear transform: exact at 48 kHz by construction, and within 0.07 dB of it at 16 kHz and 0.29 dB at 8 kHz, from 20 Hz to Nyquist.

Classes:

Name	Description
LoudnessCurve	Loudness measured over a sliding window, in LUFS.

Functions:

Name	Description
loudness	Gated integrated loudness of y, in LUFS.
loudness_over_time	Loudness measured over a sliding window -- where a take drifted.
loudness_range	Loudness range in LU, to EBU Tech 3342.
true_peak	Peak of the reconstructed waveform, in dBTP (BS.1770-4 Annex 2).
normalize_lufs	Scale y to lufs integrated loudness. Silence is returned unchanged.
match_loudness	Scale y until it is as loud as reference.

Attributes:

Name	Type	Description
MOMENTARY		The two EBU Tech 3341 meter windows, in seconds. Momentary follows
SHORT_TERM		The short-term window, in seconds (EBU Tech 3341).

BLOCK_SIZE module-attribute

BLOCK_SIZE = 0.4

Block length and hop of the gating measurement, in seconds (75% overlap).

MOMENTARY module-attribute

MOMENTARY = 0.4

The two EBU Tech 3341 meter windows, in seconds. Momentary follows syllables; short-term follows how loud a passage is.

SHORT_TERM module-attribute

SHORT_TERM = 3.0

The short-term window, in seconds (EBU Tech 3341).

CURVE_HOP module-attribute

CURVE_HOP = 0.1

Seconds between measurements on a curve -- what both meters step by.

LoudnessCurve dataclass

LoudnessCurve(
    lufs: ndarray,
    times: ndarray,
    sr: int,
    window: float,
    hop: float,
)

Loudness measured over a sliding window, in LUFS.

Parameters:

Name	Type	Description	Default
lufs	ndarray	one value per hop; -inf for a silent window, nan for one the K-weighting filter carried a NaN or inf sample into.	required
times	ndarray	centre time of each window, in seconds. A meter reports at the end of its window; a picture drawn beside a waveform has to line up with the audio that produced it, and the centre is where that is.	required

Methods:

Name	Description
loudest	(seconds, lufs) of the loudest window; nan if all silent.
quietest	(seconds, lufs) of the quietest window that is not silence.
spans_below	Where the curve sits below lufs, as (start, end) seconds.
spans_above	Where the curve sits above lufs, as (start, end) seconds.
summary	Everything above as plain numbers, for a report or a CSV row.

Attributes:

Name	Type	Description
finite	ndarray	The measurements that are not digital silence.
range_lu	float	The EBU Tech 3342 statistic over this curve, in LU.

finite property

finite: ndarray

The measurements that are not digital silence.

range_lu property

range_lu: float

The EBU Tech 3342 statistic over this curve, in LU.

It is the loudness range when the curve is short-term, which is what loudness_range builds. On a momentary curve the same arithmetic still runs and means something narrower -- how much the syllables vary -- so it is not called LRA here.

Two gates: an absolute one at -70 LUFS, and a relative one 20 LU below the power-mean of what survived it (not the integrated measurement's 10 LU). What remains is measured from its 10th to its 95th percentile, which is what stops a single door slam from setting the answer.

0.0 when nothing survives the gates: a silent clip has no range rather than an undefined one.

loudest

loudest() -> Tuple[float, float]

(seconds, lufs) of the loudest window; nan if all silent.

Source code in kudio/core/loudness.py

def loudest(self) -> Tuple[float, float]:
    """``(seconds, lufs)`` of the loudest window; ``nan`` if all silent."""
    return self._extreme(np.argmax)

quietest

quietest() -> Tuple[float, float]

(seconds, lufs) of the quietest window that is not silence.

Silence is excluded deliberately: the quietest moment of a take is almost always a pause, and "your quietest moment is the gap between two words" is not a finding.

Source code in kudio/core/loudness.py

def quietest(self) -> Tuple[float, float]:
    """``(seconds, lufs)`` of the quietest window that is not silence.

    Silence is excluded deliberately: the quietest moment of a take is
    almost always a pause, and "your quietest moment is the gap between
    two words" is not a finding.
    """
    return self._extreme(np.argmin)

spans_below

spans_below(
    lufs: float, min_seconds: float = 0.0
) -> List[Tuple[float, float]]

Where the curve sits below lufs, as (start, end) seconds.

Source code in kudio/core/loudness.py

def spans_below(self, lufs: float,
                min_seconds: float = 0.0) -> List[Tuple[float, float]]:
    """Where the curve sits below *lufs*, as ``(start, end)`` seconds."""
    return self._spans(self.lufs < lufs, min_seconds)

spans_above

spans_above(
    lufs: float, min_seconds: float = 0.0
) -> List[Tuple[float, float]]

Where the curve sits above lufs, as (start, end) seconds.

Source code in kudio/core/loudness.py

def spans_above(self, lufs: float,
                min_seconds: float = 0.0) -> List[Tuple[float, float]]:
    """Where the curve sits above *lufs*, as ``(start, end)`` seconds."""
    return self._spans(np.isfinite(self.lufs) & (self.lufs > lufs),
                       min_seconds)

summary

summary() -> dict

Everything above as plain numbers, for a report or a CSV row.

Source code in kudio/core/loudness.py

def summary(self) -> dict:
    """Everything above as plain numbers, for a report or a CSV row."""
    loud_at, loud = self.loudest()
    quiet_at, quiet = self.quietest()
    return {
        'windows': len(self),
        'range_lu': self.range_lu,
        'window': self.window,
        'max_lufs': loud,
        'max_at': loud_at,
        'min_lufs': quiet,
        'min_at': quiet_at,
        'spread_lu': (loud - quiet
                      if np.isfinite(loud) and np.isfinite(quiet)
                      else float('nan')),
    }

loudness

loudness(
    y: ndarray, sr: int, block_size: float = BLOCK_SIZE
) -> float

Gated integrated loudness of y, in LUFS.

Mono, or (samples, channels) -- channels last, the layout soundfile returns. Digital silence measures -inf; a clip with NaN or inf in it measures nan, with a warning, rather than reading as silence.

block_size is the gating block in seconds. The standard says 400 ms and that is what you want; lowering it is the only way to measure something shorter than a block, and it is then no longer a BS.1770 measurement.

>>> kudio.loudness(y, sr)
-19.4

Source code in kudio/core/loudness.py

def loudness(y: np.ndarray, sr: int, block_size: float = BLOCK_SIZE) -> float:
    """Gated integrated loudness of *y*, in LUFS.

    Mono, or ``(samples, channels)`` -- **channels last**, the layout
    ``soundfile`` returns. Digital silence measures ``-inf``; a clip with NaN
    or inf in it measures ``nan``, with a warning, rather than reading as
    silence.

    *block_size* is the gating block in seconds. The standard says 400 ms and
    that is what you want; lowering it is the only way to measure something
    shorter than a block, and it is then no longer a BS.1770 measurement.

    >>> kudio.loudness(y, sr)                       # doctest: +SKIP
    -19.4
    """
    if sr <= 0:
        raise FeatureError(f"loudness() needs a positive sample rate, got {sr}")
    data = _as_channels(y)
    if _corrupt(data, "loudness()"):
        return float('nan')
    n_channels = data.shape[1]
    if n_channels > len(_CHANNEL_WEIGHTS):
        raise FeatureError(f"loudness() handles up to {len(_CHANNEL_WEIGHTS)} "
                           f"channels, got {n_channels}")

    block = int(round(block_size * sr))
    if block < 1 or data.shape[0] < block:
        raise FeatureError(
            f"loudness() needs at least {block_size:.3f} s "
            f"({block} samples at {sr} Hz) but got {data.shape[0]} samples; "
            f"pass a smaller block_size to measure a shorter clip")

    step = max(1, int(round(block * (1.0 - _OVERLAP))))
    power = _block_power(data, int(sr), block, step)
    weights = np.array(_CHANNEL_WEIGHTS[:n_channels])

    def _level(blocks: np.ndarray) -> float:
        if blocks.size == 0:
            return float('-inf')
        total = float(np.sum(weights * np.mean(blocks, axis=0)))
        return _OFFSET + 10.0 * np.log10(total) if total > 0 else float('-inf')

    with np.errstate(divide='ignore'):
        per_block = _OFFSET + 10.0 * np.log10(power @ weights)

    above_absolute = power[per_block > _ABSOLUTE_GATE]
    if above_absolute.size == 0:
        return float('-inf')
    relative = _level(above_absolute) + _RELATIVE_GATE
    return _level(power[(per_block > _ABSOLUTE_GATE) & (per_block > relative)])

loudness_over_time

loudness_over_time(
    y: ndarray,
    sr: int,
    *,
    window: float = SHORT_TERM,
    hop: float = CURVE_HOP,
) -> LoudnessCurve

Loudness measured over a sliding window -- where a take drifted.

loudness returns one number for a whole recording, which is what you normalise against and useless for finding the moment somebody turned away from the microphone.

Parameters:

Name	Type	Description	Default
window	float	MOMENTARY (400 ms) or SHORT_TERM (3 s), the two EBU Tech 3341 meters. Momentary follows syllables; short-term follows how loud a passage is.	SHORT_TERM
hop	float	seconds between measurements; 100 ms is what the meters use. These are ungated. Gating belongs to the integrated figure, where its job is to stop pauses dragging the average down. A curve is a continuous quantity, and one with holes punched in it where the gate fired would describe a recording that stops existing between words. >>> curve = kudio.loudness_over_time(y, sr) >>> curve.quietest() (12.4, -38.2)	CURVE_HOP

Source code in kudio/core/loudness.py

def loudness_over_time(y: np.ndarray, sr: int, *,
                       window: float = SHORT_TERM,
                       hop: float = CURVE_HOP) -> LoudnessCurve:
    """Loudness measured over a sliding window -- where a take drifted.

    :func:`loudness` returns one number for a whole recording, which is what
    you normalise against and useless for finding the moment somebody turned
    away from the microphone.

    :param window: :data:`MOMENTARY` (400 ms) or :data:`SHORT_TERM` (3 s), the
        two EBU Tech 3341 meters. Momentary follows syllables; short-term
        follows how loud a passage *is*.
    :param hop: seconds between measurements; 100 ms is what the meters use.

    **These are ungated.** Gating belongs to the integrated figure, where its
    job is to stop pauses dragging the average down. A curve is a continuous
    quantity, and one with holes punched in it where the gate fired would
    describe a recording that stops existing between words.

    >>> curve = kudio.loudness_over_time(y, sr)              # doctest: +SKIP
    >>> curve.quietest()                                     # doctest: +SKIP
    (12.4, -38.2)
    """
    if sr <= 0:
        raise FeatureError(f"loudness_over_time() needs a positive rate, got {sr}")
    if window <= 0 or hop <= 0:
        raise FeatureError(f"window and hop must be positive, got "
                           f"window={window}, hop={hop}")
    data = _as_channels(y)
    n_channels = data.shape[1]
    if n_channels > len(_CHANNEL_WEIGHTS):
        raise FeatureError(f"loudness_over_time() handles up to "
                           f"{len(_CHANNEL_WEIGHTS)} channels, got {n_channels}")

    block = int(round(window * sr))
    if block < 1 or data.shape[0] < block:
        raise FeatureError(
            f"loudness_over_time() needs at least {window:g} s "
            f"({block} samples at {sr} Hz) but got {data.shape[0]}; "
            f"pass a shorter window to measure a shorter clip")

    step = max(1, int(round(hop * sr)))
    power = _block_power(data, int(sr), block, step)
    weights = np.array(_CHANNEL_WEIGHTS[:n_channels])
    with np.errstate(divide='ignore'):
        values = _OFFSET + 10.0 * np.log10(power @ weights)
    _corrupt(data, "loudness_over_time()")
    starts = np.arange(len(values)) * step
    times = (starts + block / 2.0) / sr
    return LoudnessCurve(values, times, int(sr), float(window), float(hop))

loudness_range

loudness_range(y: ndarray, sr: int) -> float

Loudness range in LU, to EBU Tech 3342.

How far the loud parts sit above the quiet parts once the silence and the outliers are out of the way. A voice recording with a consistent delivery lands near 3-6 LU; one where somebody walked away from the microphone does not, and the integrated figure cannot tell you which you have.

It is the 10th-to-95th percentile spread of the short-term loudness, after two gates: an absolute one at -70 LUFS, and a relative one 20 LU below the power-mean of what survived it. The percentiles are what keep a single door slam from setting the answer, and the -20 LU gate -- not the integrated measurement's -10 -- is the one Tech 3342 specifies.

Returns 0.0 when nothing survives the gates; a silent or very short clip has no range rather than an undefined one. A clip with NaN or inf in it returns nan, with a warning.

If you already have the short-term curve, read LoudnessCurve.range_lu instead -- this builds one and asks it, and measuring the curve twice is the expensive half of both.

>>> kudio.loudness_range(y, sr)
5.7

Source code in kudio/core/loudness.py

def loudness_range(y: np.ndarray, sr: int) -> float:
    """Loudness range in LU, to EBU Tech 3342.

    How far the loud parts sit above the quiet parts once the silence and the
    outliers are out of the way. A voice recording with a consistent delivery
    lands near 3-6 LU; one where somebody walked away from the microphone does
    not, and the integrated figure cannot tell you which you have.

    It is the 10th-to-95th percentile spread of the **short-term** loudness,
    after two gates: an absolute one at -70 LUFS, and a relative one 20 LU
    below the power-mean of what survived it. The percentiles are what keep a
    single door slam from setting the answer, and the -20 LU gate -- not the
    integrated measurement's -10 -- is the one Tech 3342 specifies.

    Returns ``0.0`` when nothing survives the gates; a silent or very short
    clip has no range rather than an undefined one. A clip with NaN or inf in
    it returns ``nan``, with a warning.

    If you already have the short-term curve, read
    :attr:`LoudnessCurve.range_lu` instead -- this builds one and asks it, and
    measuring the curve twice is the expensive half of both.

    >>> kudio.loudness_range(y, sr)                          # doctest: +SKIP
    5.7
    """
    curve = loudness_over_time(y, sr, window=SHORT_TERM, hop=CURVE_HOP)
    return float('nan') if _unmeasured(curve.lufs) else curve.range_lu

true_peak

true_peak(
    y: ndarray, sr: int, *, oversample: Optional[int] = None
) -> float

Peak of the reconstructed waveform, in dBTP (BS.1770-4 Annex 2).

The largest sample is not the largest value the signal reaches. Samples are points on a curve that passes between them, and the curve overshoots: a full-scale sine at a quarter of the sample rate, sampled at the zero crossings and the points between, reads far below the peak it actually has. Anything that reconstructs the waveform -- a converter, a lossy encoder, a resampler -- meets the real value and clips on it, which is why broadcast specifications are written in dBTP and not dBFS.

Measured by oversampling and taking the maximum. oversample defaults to whatever brings the rate to at least 192 kHz, which is what the standard asks for; at 48 kHz that is the familiar 4x.

>>> kudio.true_peak(y, sr)
0.8

Source code in kudio/core/loudness.py

def true_peak(y: np.ndarray, sr: int, *,
              oversample: Optional[int] = None) -> float:
    """Peak of the *reconstructed* waveform, in dBTP (BS.1770-4 Annex 2).

    The largest sample is not the largest value the signal reaches. Samples are
    points on a curve that passes between them, and the curve overshoots: a
    full-scale sine at a quarter of the sample rate, sampled at the zero
    crossings and the points between, reads far below the peak it actually
    has. Anything that reconstructs the waveform -- a converter, a lossy
    encoder, a resampler -- meets the real value and clips on it, which is why
    broadcast specifications are written in dBTP and not dBFS.

    Measured by oversampling and taking the maximum. *oversample* defaults to
    whatever brings the rate to at least 192 kHz, which is what the standard
    asks for; at 48 kHz that is the familiar 4x.

    >>> kudio.true_peak(y, sr)                               # doctest: +SKIP
    0.8
    """
    if sr <= 0:
        raise FeatureError(f"true_peak() needs a positive sample rate, got {sr}")
    data = _as_channels(y)
    if data.shape[0] == 0:
        return float('-inf')
    if _corrupt(data, "true_peak()", "true peak"):
        return float('nan')
    factor = int(oversample) if oversample is not None else max(
        1, int(np.ceil(_TRUE_PEAK_SR / sr)))
    if factor < 1:
        raise FeatureError(f"oversample must be >= 1, got {oversample}")

    peak = _oversampled_peak(data, factor)
    return 20.0 * float(np.log10(peak)) if peak > 0 else float('-inf')

normalize_lufs

normalize_lufs(
    y: ndarray,
    sr: int,
    lufs: float = -23.0,
    block_size: float = BLOCK_SIZE,
) -> ndarray

Scale y to lufs integrated loudness. Silence is returned unchanged.

-23 LUFS is the EBU R 128 broadcast target. This applies a gain and nothing else, so the result can exceed full scale; that is warned about rather than silently limited, because clamping here would change the loudness you just asked for. Integer audio comes back in its own type, rounded, and there the clipping has already happened.

Raises:

Type	Description
FeatureError	for a clip with NaN or inf in it, which has no loudness to scale from. >>> quiet = kudio.normalize_lufs(y, sr, lufs=-23.0)

Source code in kudio/core/loudness.py

def normalize_lufs(y: np.ndarray, sr: int, lufs: float = -23.0,
                   block_size: float = BLOCK_SIZE) -> np.ndarray:
    """Scale *y* to *lufs* integrated loudness. Silence is returned unchanged.

    -23 LUFS is the EBU R 128 broadcast target. This applies a gain and nothing
    else, so the result can exceed full scale; that is warned about rather than
    silently limited, because clamping here would change the loudness you just
    asked for. Integer audio comes back in its own type, rounded, and there
    the clipping has already happened.

    :raises FeatureError: for a clip with NaN or inf in it, which has no
        loudness to scale from.

    >>> quiet = kudio.normalize_lufs(y, sr, lufs=-23.0)      # doctest: +SKIP
    """
    current = loudness(y, sr, block_size=block_size)
    if np.isnan(current):
        raise FeatureError("normalize_lufs() got a clip with samples that are "
                           "not finite; repair it first")
    if not np.isfinite(current):
        return np.asarray(y)
    x, back = to_float(y)
    out = x * float(10 ** ((lufs - current) / 20.0))
    peak = float(np.max(np.abs(out))) if out.size else 0.0
    if peak > 1.0:
        fate = ("integer audio is clipped there"
                if np.issubdtype(np.asarray(y).dtype, np.integer)
                else "the signal will clip when written to an integer format")
        warnings.warn(f"normalize_lufs: {lufs:.1f} LUFS puts the peak at "
                      f"{peak:.2f} ({20 * np.log10(peak):.1f} dBFS over); "
                      f"{fate}", UserWarning, stacklevel=2)
    return back(out)

match_loudness

match_loudness(
    y: ndarray,
    reference: ndarray,
    sr: int,
    reference_sr: Optional[int] = None,
    block_size: float = BLOCK_SIZE,
) -> ndarray

Scale y until it is as loud as reference.

The primitive behind a fair A/B: comparing a processed clip against its source tells you about the processing only once the level difference is out of the way, since the louder of two clips is reliably preferred regardless of quality. Returns y unchanged when either side is silent, and integer audio in its own type, rounded and clipped.

Raises:

Type	Description
FeatureError	when either clip has NaN or inf in it, and so no loudness to match. reference_sr defaults to sr. Pass it when the two were captured at different rates -- loudness is a property of the sound, not of the sampling, so the two are still comparable. >>> fair = kudio.match_loudness(enhanced, original, sr)

Source code in kudio/core/loudness.py

def match_loudness(y: np.ndarray, reference: np.ndarray, sr: int,
                   reference_sr: Optional[int] = None,
                   block_size: float = BLOCK_SIZE) -> np.ndarray:
    """Scale *y* until it is as loud as *reference*.

    The primitive behind a fair A/B: comparing a processed clip against its
    source tells you about the processing only once the level difference is
    out of the way, since the louder of two clips is reliably preferred
    regardless of quality. Returns *y* unchanged when either side is silent,
    and integer audio in its own type, rounded and clipped.

    :raises FeatureError: when either clip has NaN or inf in it, and so no
        loudness to match.

    *reference_sr* defaults to *sr*. Pass it when the two were captured at
    different rates -- loudness is a property of the sound, not of the
    sampling, so the two are still comparable.

    >>> fair = kudio.match_loudness(enhanced, original, sr)  # doctest: +SKIP
    """
    target = loudness(reference, reference_sr or sr, block_size=block_size)
    current = loudness(y, sr, block_size=block_size)
    if np.isnan(target) or np.isnan(current):
        side = "reference" if np.isnan(target) else "clip"
        raise FeatureError(f"match_loudness() got a {side} with samples that "
                           f"are not finite; repair it first")
    if not (np.isfinite(target) and np.isfinite(current)):
        return np.asarray(y)
    x, back = to_float(y)
    return back(x * float(10 ** ((target - current) / 20.0)))

kudio.core.dnsmos

DNSMOS: a predicted listener opinion, with no clean reference.

kudio.audio_report answers "is this recording technically sound". It cannot answer "does it sound good" — that is a judgement, and the only reference-free way to get one without a listening panel is a model trained on one. DNSMOS (ITU-T P.835) is that model: it returns three scores on the usual 1–5 MOS scale.

• SIG — the speech itself, ignoring the background.
• BAK — how unobtrusive the background is.
• OVRL — overall quality.

The interesting part for enhancement work is that SIG and BAK move in opposite directions: aggressive denoising raises BAK and lowers SIG, because it eats the speech along with the noise. A single number hides that trade; three do not.

kudio does not ship the model. The weights belong to Microsoft's DNS-Challenge and are distributed there; bundling ~2 MB of someone else's binary in a library that is otherwise pure source is not a thing to do quietly. Point model_dir at a checkout:

>>> kudio.dnsmos(y, sr, model_dir="DNS-Challenge/DNSMOS")
DnsmosScore(sig=3.42, bak=3.91, ovrl=3.15, segments=4)

Needs pip install kudio[dnsmos] for onnxruntime.

Classes:

Name	Description
DnsmosScore	Three P.835 opinion scores, each on the 1–5 MOS scale.

Functions:

Name	Description
find_dnsmos_model	Locate sig_bak_ovr.onnx.
dnsmos	Predict P.835 opinion scores for y.

MODEL_SR module-attribute

MODEL_SR = 16000

The model was trained at 16 kHz and on this segment length, both of which are part of the model rather than choices we are free to make.

MODEL_NAMES module-attribute

MODEL_NAMES = ('sig_bak_ovr.onnx',)

The P.835 model's file name in the DNS-Challenge distribution. The model_v8.onnx beside it is the P.808 model, which gives one score, not three, and is not looked for.

ENV_VAR module-attribute

ENV_VAR = 'KUDIO_DNSMOS_DIR'

The environment variable find_dnsmos_model reads when no model_dir is given: the folder holding the model, or the model itself.

POLYFIT module-attribute

POLYFIT = {
    "sig": (-0.08397278, 1.22083953, 0.0052439),
    "bak": (-0.13166888, 1.60915514, -0.39604546),
    "ovrl": (-0.06766283, 1.11546468, 0.04602535),
}

Published P.835 mapping from the network's raw outputs onto the MOS scale (DNSMOS P.835, non-personalised). They are part of the released model, not a tuning of ours -- which also means kudio cannot verify them without the weights. polyfit=False returns the raw outputs if you would rather not rely on them, and a polyfit.json in model_dir overrides them.

DnsmosScore dataclass

DnsmosScore(
    sig: float, bak: float, ovrl: float, segments: int
)

Three P.835 opinion scores, each on the 1–5 MOS scale.

Methods:

Name	Description
summary	One line of plain words about what the numbers mean.

Attributes:

Name	Type	Description
sig	float	quality of the speech signal alone
bak	float	how unobtrusive the background is
ovrl	float	overall quality
segments	int	how many 9-second windows were averaged

sig instance-attribute

sig: float

quality of the speech signal alone

bak instance-attribute

bak: float

how unobtrusive the background is

ovrl instance-attribute

ovrl: float

overall quality

segments instance-attribute

segments: int

how many 9-second windows were averaged

summary

summary() -> str

One line of plain words about what the numbers mean.

Source code in kudio/core/dnsmos.py

def summary(self) -> str:
    """One line of plain words about what the numbers mean."""
    if self.ovrl >= 4.0:
        verdict = "clean"
    elif self.ovrl >= 3.0:
        verdict = "usable"
    elif self.ovrl >= 2.0:
        verdict = "poor"
    else:
        verdict = "bad"
    note = ""
    if self.bak - self.sig > 0.7:
        note = " — the background is cleaner than the speech, which is what "\
               "over-aggressive denoising looks like"
    elif self.sig - self.bak > 0.7:
        note = " — the speech survived but the background is still intrusive"
    return f"{verdict} (OVRL {self.ovrl:.2f}){note}"

find_dnsmos_model

find_dnsmos_model(
    model_dir: Optional[PathLike] = None,
) -> Path

Locate sig_bak_ovr.onnx.

Looks at model_dir, then at $KUDIO_DNSMOS_DIR. Either may be the directory holding the file or the file itself.

Raises:

Type	Description
FeatureError	with instructions, when nothing is found. Being told where to get the model beats a FileNotFoundError on a path you did not choose.

Source code in kudio/core/dnsmos.py

def find_dnsmos_model(model_dir: Optional[PathLike] = None) -> Path:
    """Locate ``sig_bak_ovr.onnx``.

    Looks at *model_dir*, then at ``$KUDIO_DNSMOS_DIR``. Either may be the
    directory holding the file or the file itself.

    :raises FeatureError: with instructions, when nothing is found. Being told
        where to get the model beats a FileNotFoundError on a path you did not
        choose.
    """
    candidates = [model_dir, os.environ.get(ENV_VAR)]
    for candidate in candidates:
        if not candidate:
            continue
        path = Path(candidate)
        if path.is_file():
            return path
        if path.is_dir():
            for name in MODEL_NAMES:
                found = path / name
                if found.is_file():
                    return found
            # the personalised model in pDNSMOS/ shares the file name but not
            # the mapping, so the plain one comes first when both are there
            deeper = sorted(path.rglob("sig_bak_ovr.onnx"),
                            key=lambda p: ("pDNSMOS" in p.parts, str(p)))
            if deeper:
                return deeper[0]
    raise FeatureError(
        "DNSMOS model not found. kudio does not ship it — clone "
        "https://github.com/microsoft/DNS-Challenge and pass "
        "model_dir='DNS-Challenge/DNSMOS', or set "
        f"{ENV_VAR} to that directory.")

dnsmos

dnsmos(
    y: ndarray,
    sr: int,
    model_dir: Optional[PathLike] = None,
    *,
    polyfit: bool = True,
    session: Optional[Any] = None,
) -> DnsmosScore

Predict P.835 opinion scores for y.

Mono only. Audio at any rate is resampled to the 16 kHz the model was trained at — the rate is a property of the model, not a preference.

Parameters:

Name	Type	Description	Default
model_dir	Optional[PathLike]	directory (or file) holding sig_bak_ovr.onnx. See find_dnsmos_model for the search order.	None
polyfit	bool	map the network's raw outputs onto the MOS scale using the published coefficients. False returns the raw outputs, which are monotonic in quality but not on the 1–5 scale.	True
session	Optional[Any]	a pre-built inference session, so a batch job opens the model once instead of per file.	None

Raises:

Type	Description
DependencyError	when onnxruntime is not installed.
FeatureError	on non-mono audio, an empty clip, or a missing model.

Source code in kudio/core/dnsmos.py

def dnsmos(y: np.ndarray, sr: int, model_dir: Optional[PathLike] = None, *,
           polyfit: bool = True, session: Optional[Any] = None) -> DnsmosScore:
    """Predict P.835 opinion scores for *y*.

    Mono only. Audio at any rate is resampled to the 16 kHz the model was
    trained at — the rate is a property of the model, not a preference.

    :param model_dir: directory (or file) holding ``sig_bak_ovr.onnx``. See
        :func:`find_dnsmos_model` for the search order.
    :param polyfit: map the network's raw outputs onto the MOS scale using the
        published coefficients. ``False`` returns the raw outputs, which are
        monotonic in quality but not on the 1–5 scale.
    :param session: a pre-built inference session, so a batch job opens the
        model once instead of per file.

    :raises DependencyError: when onnxruntime is not installed.
    :raises FeatureError: on non-mono audio, an empty clip, or a missing model.
    """
    if sr <= 0:
        raise FeatureError(f"dnsmos() needs a positive sample rate, got {sr}")
    y = np.ascontiguousarray(np.asarray(as_float(y), dtype=np.float32).squeeze())
    if y.ndim != 1:
        raise FeatureError(f"dnsmos() takes mono audio, got shape {y.shape}")
    if y.size == 0:
        raise FeatureError("dnsmos() got an empty waveform")

    if sr != MODEL_SR:
        from kudio.core.io import resample
        y = resample(y, sr, MODEL_SR).astype(np.float32, copy=False)

    coefficients = dict(POLYFIT)
    if session is None:
        model_path = find_dnsmos_model(model_dir)
        if "pDNSMOS" in model_path.parts:
            warnings.warn(f"{model_path} is the personalised DNSMOS model, which "
                          f"the published mapping kudio applies was not fitted "
                          f"for; pass polyfit=False for its raw outputs, or a "
                          f"polyfit.json beside it", UserWarning, stacklevel=2)
        session = _session(model_path)
        coefficients = _load_polyfit(model_path)

    input_name = session.get_inputs()[0].name
    raw = []
    for segment in _segments(y, MODEL_SR):
        features = segment.astype(np.float32)[np.newaxis, :]
        out = session.run(None, {input_name: features})
        raw.append(np.asarray(out[0], dtype=np.float64).reshape(-1)[:3])
    if not raw:                                  # unreachable: _segments tiles
        raise FeatureError("dnsmos() produced no windows")

    windows = np.vstack(raw)
    if polyfit:
        # each window mapped, then the mean: the order DNS-Challenge's own
        # dnsmos_local.py uses, and with a quadratic the two orders differ
        windows = np.column_stack([
            np.polyval(coefficients[key], windows[:, i])
            for i, key in enumerate(("sig", "bak", "ovrl"))])
    sig, bak, ovrl = np.mean(windows, axis=0)
    return DnsmosScore(sig=float(sig), bak=float(bak), ovrl=float(ovrl),
                       segments=len(raw))